[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:06 dist-test-slave-gtr8 krb5kdc[8806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:06 dist-test-slave-gtr8 krb5kdc[8806](info): set up 2 sockets
Sep 28 06:40:06 dist-test-slave-gtr8 krb5kdc[8806](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260928 06:40:06.943611 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:07.068293 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:42665
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:42665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:07.175560 8819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:07.175814 8819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:07.175870 8819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:07.179790 8819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:07.179847 8819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:07.179872 8819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:07.179893 8819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:07.179919 8819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:07.185098 8819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:42665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:42665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:07.186151 8819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:07.187229 8819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:07.196095 8825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:07.196180 8819 server_base.cc:1034] running on GCE node
W20260928 06:40:07.196178 8824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:07.196271 8827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:07.196925 8819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:07.198092 8819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:07.199230 8819 hybrid_clock.cc:648] HybridClock initialized: now 1790577607199209 us; error 33 us; skew 500 ppm
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:07.201959 8819 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:07.202816 8819 webserver.cc:460] Webserver started at http://127.8.151.254:35251/ using document root <none> and password file <none>
I20260928 06:40:07.203442 8819 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:07.203507 8819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:07.203696 8819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:07.205781 8819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "ea694070127c4bb8a14eb7b3759c6fed"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "afe7c41c2824588de2a60a0bbfbdef19"
server_key_iv: "de36a5263f90fa3cc7fac58493d36863"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.206283 8819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "ea694070127c4bb8a14eb7b3759c6fed"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "afe7c41c2824588de2a60a0bbfbdef19"
server_key_iv: "de36a5263f90fa3cc7fac58493d36863"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.210028 8819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:40:07.212337 8834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:07.213539 8819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:07.213641 8819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "ea694070127c4bb8a14eb7b3759c6fed"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "afe7c41c2824588de2a60a0bbfbdef19"
server_key_iv: "de36a5263f90fa3cc7fac58493d36863"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.213742 8819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:07.230396 8819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:07.233752 8819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:07.233925 8819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:07.240794 8819 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:42665
I20260928 06:40:07.240820 8886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:42665 every 8 connection(s)
I20260928 06:40:07.241747 8819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:07.244467 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 8819
I20260928 06:40:07.244619 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:07.244609 8887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:07.245867 8799 external_mini_cluster.cc:1428] Setting key 85cdee36020e72a7c88c20219597c533
I20260928 06:40:07.250679 8887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed: Bootstrap starting.
I20260928 06:40:07.252902 8887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:07.253616 8887 log.cc:824] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:07.255457 8887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed: No bootstrap required, opened a new log
I20260928 06:40:07.258890 8887 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea694070127c4bb8a14eb7b3759c6fed" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42665 } }
I20260928 06:40:07.259116 8887 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:07.259182 8887 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea694070127c4bb8a14eb7b3759c6fed, State: Initialized, Role: FOLLOWER
I20260928 06:40:07.259615 8887 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea694070127c4bb8a14eb7b3759c6fed" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42665 } }
I20260928 06:40:07.259747 8887 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:07.259809 8887 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:07.259902 8887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:07.260830 8887 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea694070127c4bb8a14eb7b3759c6fed" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42665 } }
I20260928 06:40:07.261150 8887 leader_election.cc:302] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea694070127c4bb8a14eb7b3759c6fed; no voters:
I20260928 06:40:07.261412 8887 leader_election.cc:288] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:07.261610 8892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:07.261833 8892 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [term 1 LEADER]: Becoming Leader. State: Replica: ea694070127c4bb8a14eb7b3759c6fed, State: Running, Role: LEADER
I20260928 06:40:07.262173 8892 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea694070127c4bb8a14eb7b3759c6fed" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42665 } }
I20260928 06:40:07.262602 8887 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [sys.catalog]: configured and running, proceeding with master startup.
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:07.263722 8893 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea694070127c4bb8a14eb7b3759c6fed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea694070127c4bb8a14eb7b3759c6fed" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42665 } } }
I20260928 06:40:07.263873 8893 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:07.263887 8894 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea694070127c4bb8a14eb7b3759c6fed. Latest consensus state: current_term: 1 leader_uuid: "ea694070127c4bb8a14eb7b3759c6fed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea694070127c4bb8a14eb7b3759c6fed" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42665 } } }
I20260928 06:40:07.263970 8894 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:07.264284 8901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:07.267447 8901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:07.270105 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:07.249049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.8.151.254:42665)
0928 06:40:07.249473 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:07.249488 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:07.250522 (+ 1034us) server_negotiation.cc:407] Connection header received
0928 06:40:07.253623 (+ 3101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:07.253638 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:07.253979 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:07.254281 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:07.256492 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.257442 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.258586 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.259043 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.259209 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:07.264165 (+ 4956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:07.264172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:07.264189 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:07.264221 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:07.266307 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.266941 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.266947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.266953 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.267091 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.267527 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.267530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.267532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.268028 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:07.268247 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:07.268908 (+ 661us) server_negotiation.cc:299] Negotiation successful
0928 06:40:07.269206 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":122,"threads_started":1}
I20260928 06:40:07.273089 8901 catalog_manager.cc:1347] Generated new cluster ID: 85ab40330675422aa5041cde43f5b57a
I20260928 06:40:07.273155 8901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:07.281148 8901 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:07.282281 8901 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:07.291357 8901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea694070127c4bb8a14eb7b3759c6fed: Generated new TSK 0
I20260928 06:40:07.292069 8901 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:40:07.324306 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42665
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:07.433827 8913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:07.434099 8913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:07.434161 8913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:07.438161 8913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:07.438222 8913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:07.438282 8913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:07.443675 8913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42665
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:07.444689 8913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:07.445672 8913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:07.455014 8918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:07.455009 8921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:07.455014 8919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:07.455261 8913 server_base.cc:1034] running on GCE node
I20260928 06:40:07.455754 8913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:07.456357 8913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:07.457541 8913 hybrid_clock.cc:648] HybridClock initialized: now 1790577607457519 us; error 33 us; skew 500 ppm
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:07.460264 8913 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:40:07.461088 8913 webserver.cc:460] Webserver started at http://127.8.151.193:34079/ using document root <none> and password file <none>
I20260928 06:40:07.461620 8913 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:07.461676 8913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:07.461838 8913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:07.463940 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "b3c742a527324dc3b03836aea60d8d2c"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "6ba3087eaab803d0c46df35791d1bc7b"
server_key_iv: "5b181d872dedbc578982b9d2949a288d"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.464468 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "b3c742a527324dc3b03836aea60d8d2c"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "6ba3087eaab803d0c46df35791d1bc7b"
server_key_iv: "5b181d872dedbc578982b9d2949a288d"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.468066 8913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:40:07.470453 8928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:07.471748 8913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:40:07.471858 8913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "b3c742a527324dc3b03836aea60d8d2c"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "6ba3087eaab803d0c46df35791d1bc7b"
server_key_iv: "5b181d872dedbc578982b9d2949a288d"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.471969 8913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:07.482259 8913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:07.485291 8913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:07.485476 8913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:07.486127 8913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:07.486896 8913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:07.486948 8913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:07.486986 8913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:07.487005 8913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:07.495834 8913 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:39363
I20260928 06:40:07.495930 9041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:39363 every 8 connection(s)
I20260928 06:40:07.496886 8913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:07.500285 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 8913
I20260928 06:40:07.500388 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:07.500622 8799 external_mini_cluster.cc:1428] Setting key 41892254809229faee47d97dbbfb9651
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:40:07.511024 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:07.499285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:43923 (local address 127.8.151.254:42665)
0928 06:40:07.500385 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:07.500390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:07.500409 (+ 19us) server_negotiation.cc:407] Connection header received
0928 06:40:07.501535 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:07.501538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:07.501609 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:07.501683 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:07.502891 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.503543 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.504645 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.504869 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.504954 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:07.507535 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:07.507539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:07.507541 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:07.507566 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:07.508921 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.509447 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.509452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.509454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.509535 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.509859 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.509863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.509865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.510100 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:07.510205 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:07.510712 (+ 507us) server_negotiation.cc:299] Negotiation successful
0928 06:40:07.510841 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:40:07.511982 9044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:07.499416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42665 (local address 127.8.151.193:43923)
0928 06:40:07.499901 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:07.499945 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:07.501288 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:07.501858 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:07.501866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:07.502191 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:07.502668 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:07.502681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.503662 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.503666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:07.504532 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:07.504541 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.505006 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.505010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:07.505161 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:07.505684 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:07.505700 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:07.507360 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:07.509045 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:07.509051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:07.509067 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:07.509346 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:07.509622 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:07.509626 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:07.509628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:07.509767 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:07.510224 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:07.510232 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:07.510477 (+ 245us) client_negotiation.cc:769] Sending connection context
0928 06:40:07.510654 (+ 177us) client_negotiation.cc:240] Negotiation successful
0928 06:40:07.511024 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":120,"threads_started":1}
I20260928 06:40:07.513206 9042 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42665
I20260928 06:40:07.513491 9042 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:07.513981 9042 heartbeater.cc:511] Master 127.8.151.254:42665 requested a full tablet report, sending...
I20260928 06:40:07.515503 8851 ts_manager.cc:194] Registered new tserver with Master: b3c742a527324dc3b03836aea60d8d2c (127.8.151.193:39363)
I20260928 06:40:07.516714 8851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:43923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:40:07.526365 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42665
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:07.637298 9047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:07.637677 9047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:07.637756 9047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:07.643855 9047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:07.643914 9047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:07.643975 9047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:07.649348 9047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42665
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:07.650338 9047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:07.651414 9047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:07.660653 9053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:07.660713 9052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:07.660642 9055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:07.660758 9047 server_base.cc:1034] running on GCE node
I20260928 06:40:07.661285 9047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:07.661928 9047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:07.663112 9047 hybrid_clock.cc:648] HybridClock initialized: now 1790577607663095 us; error 34 us; skew 500 ppm
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:07.665804 9047 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:40:07.666632 9047 webserver.cc:460] Webserver started at http://127.8.151.194:41979/ using document root <none> and password file <none>
I20260928 06:40:07.667196 9047 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:07.667265 9047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:07.667461 9047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:07.669423 9047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "62ea2e557e064e8889469460a78a8061"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "6a8cb1f389b01e945c44a068db9c4de4"
server_key_iv: "de60b4785857199b31582d9dfd85dacf"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.669940 9047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "62ea2e557e064e8889469460a78a8061"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "6a8cb1f389b01e945c44a068db9c4de4"
server_key_iv: "de60b4785857199b31582d9dfd85dacf"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.673568 9047 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:07.675985 9062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:07.677310 9047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:07.677417 9047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "62ea2e557e064e8889469460a78a8061"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "6a8cb1f389b01e945c44a068db9c4de4"
server_key_iv: "de60b4785857199b31582d9dfd85dacf"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.677516 9047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:07.697242 9047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:07.700168 9047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:07.700345 9047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:07.700960 9047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:07.701835 9047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:07.701880 9047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:07.701921 9047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:07.701949 9047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:07.710469 9047 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:41315
I20260928 06:40:07.710486 9175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:41315 every 8 connection(s)
I20260928 06:40:07.711529 9047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:07.712245 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 9047
I20260928 06:40:07.712342 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:07.712580 8799 external_mini_cluster.cc:1428] Setting key 40a69bd9a39a34be766e8a42f1b667ce
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
I20260928 06:40:07.724823 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:07.713318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:52853 (local address 127.8.151.254:42665)
0928 06:40:07.713430 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:07.713433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:07.714432 (+ 999us) server_negotiation.cc:407] Connection header received
0928 06:40:07.715802 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:07.715806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:07.715860 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:07.715931 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:07.717104 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.717687 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.718822 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.719052 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.719162 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:07.721546 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:07.721550 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:07.721553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:07.721578 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:07.722744 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.723208 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.723212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.723214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.723291 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.723712 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.723715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.723717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.723897 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:07.723988 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:07.724492 (+ 504us) server_negotiation.cc:299] Negotiation successful
0928 06:40:07.724658 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 06:40:07.725636 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:07.713834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42665 (local address 127.8.151.194:52853)
0928 06:40:07.714300 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:07.714343 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:07.715610 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:07.716095 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:07.716102 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:07.716445 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:07.716972 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:07.716989 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.717795 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.717798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:07.718707 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:07.718716 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.719209 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.719213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:07.719363 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:07.719976 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:07.719998 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:07.721383 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:07.722840 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:07.722846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:07.722860 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:07.723092 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:07.723400 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:07.723403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:07.723405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:07.723598 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:07.724002 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:07.724008 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:07.724258 (+ 250us) client_negotiation.cc:769] Sending connection context
0928 06:40:07.724415 (+ 157us) client_negotiation.cc:240] Negotiation successful
0928 06:40:07.724734 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":120,"threads_started":1}
I20260928 06:40:07.726857 9176 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42665
I20260928 06:40:07.727126 9176 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:07.727702 9176 heartbeater.cc:511] Master 127.8.151.254:42665 requested a full tablet report, sending...
I20260928 06:40:07.728754 8851 ts_manager.cc:194] Registered new tserver with Master: 62ea2e557e064e8889469460a78a8061 (127.8.151.194:41315)
I20260928 06:40:07.729509 8851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:52853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:40:07.738736 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42665
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:07.847796 9181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:07.848055 9181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:07.848098 9181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:07.852036 9181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:07.852092 9181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:07.852157 9181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:07.857442 9181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42665
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:07.858443 9181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:07.859472 9181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:07.868705 9186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:07.868656 9187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:07.868811 9181 server_base.cc:1034] running on GCE node
W20260928 06:40:07.868662 9189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:07.869264 9181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:07.869865 9181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:07.871016 9181 hybrid_clock.cc:648] HybridClock initialized: now 1790577607870997 us; error 31 us; skew 500 ppm
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:07.873618 9181 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:40:07.874477 9181 webserver.cc:460] Webserver started at http://127.8.151.195:44793/ using document root <none> and password file <none>
I20260928 06:40:07.874989 9181 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:07.875048 9181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:07.875283 9181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:07.877291 9181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "06addb29f6214def89dffa546a788546"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "b9e5c3b17839885df0e18d45c2d9e8a6"
server_key_iv: "c055f7c58649655cccb02d3b1a4e9358"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.877810 9181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "06addb29f6214def89dffa546a788546"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "b9e5c3b17839885df0e18d45c2d9e8a6"
server_key_iv: "c055f7c58649655cccb02d3b1a4e9358"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.881436 9181 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260928 06:40:07.883754 9196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:07.885036 9181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260928 06:40:07.885130 9181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "06addb29f6214def89dffa546a788546"
format_stamp: "Formatted at 2026-09-28 06:40:07 on dist-test-slave-gtr8"
server_key: "b9e5c3b17839885df0e18d45c2d9e8a6"
server_key_iv: "c055f7c58649655cccb02d3b1a4e9358"
server_key_version: "encryptionkey@0"
I20260928 06:40:07.885228 9181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:07.901834 9181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:07.904678 9181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:07.904839 9181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:07.905395 9181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:07.906172 9181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:07.906215 9181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:07.906256 9181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:07.906284 9181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:07.914373 9181 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:35637
I20260928 06:40:07.914402 9309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:35637 every 8 connection(s)
I20260928 06:40:07.915319 9181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:07.925060 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 9181
I20260928 06:40:07.925153 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:07.925367 8799 external_mini_cluster.cc:1428] Setting key 93cfe99b5213a277dacba76fe8f3c28c
I20260928 06:40:07.928833 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:07.917143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:58235 (local address 127.8.151.254:42665)
0928 06:40:07.917304 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:07.917308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:07.918143 (+ 835us) server_negotiation.cc:407] Connection header received
0928 06:40:07.919494 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:07.919497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:07.919557 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:07.919626 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:07.920813 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.921338 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.922460 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.922677 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.922764 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:07.925224 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:07.925228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:07.925230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:07.925255 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:07.926785 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.927323 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.927326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.927328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.927417 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.927813 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.927817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.927818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.928016 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:07.928092 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:07.928613 (+ 521us) server_negotiation.cc:299] Negotiation successful
0928 06:40:07.928714 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 06:40:07.929553 9312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:07.917551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42665 (local address 127.8.151.195:58235)
0928 06:40:07.917998 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:07.918041 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:07.919282 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:07.919806 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:07.919813 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:07.920172 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:07.920659 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:07.920672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.921455 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.921457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:07.922342 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:07.922351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.922818 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.922822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:07.922976 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:07.923571 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:07.923586 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:07.925055 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:07.926908 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:07.926914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:07.926930 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:07.927239 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:07.927557 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:07.927560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:07.927562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:07.927717 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:07.928102 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:07.928107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:07.928336 (+ 229us) client_negotiation.cc:769] Sending connection context
0928 06:40:07.928515 (+ 179us) client_negotiation.cc:240] Negotiation successful
0928 06:40:07.928765 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":100,"threads_started":1}
I20260928 06:40:07.930526 9310 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42665
I20260928 06:40:07.930742 9310 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:07.931239 9310 heartbeater.cc:511] Master 127.8.151.254:42665 requested a full tablet report, sending...
I20260928 06:40:07.932166 8851 ts_manager.cc:194] Registered new tserver with Master: 06addb29f6214def89dffa546a788546 (127.8.151.195:35637)
I20260928 06:40:07.932683 8851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:58235
I20260928 06:40:07.939796 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:40:07.965252 9317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:07.955719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.8.151.193:39363)
0928 06:40:07.956060 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:07.956070 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:07.956096 (+ 26us) server_negotiation.cc:407] Connection header received
0928 06:40:07.956144 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:07.956149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:07.956333 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:07.956461 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:07.957174 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.958071 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.958942 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.959381 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.959464 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:07.961776 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:07.961784 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:07.961796 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:07.961819 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:07.963543 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.963953 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.963961 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.963962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.964068 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.964315 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.964318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.964320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.964556 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:07.964745 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:07.964951 (+ 206us) server_negotiation.cc:299] Negotiation successful
0928 06:40:07.965106 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":118,"threads_started":1}
W20260928 06:40:07.966189 8956 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38204
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 06:40:07 dist-test-slave-gtr8 krb5kdc[8806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:40:07.991119 9317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:07.982644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38220 (local address 127.8.151.193:39363)
0928 06:40:07.982805 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:07.982808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:07.982900 (+ 92us) server_negotiation.cc:407] Connection header received
0928 06:40:07.983017 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:07.983020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:07.983065 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:07.983173 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:07.983849 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.984379 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.985214 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:07.985460 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:07.985567 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:07.987881 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:07.987884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:07.987886 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:07.987910 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:07.989409 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.989995 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.989998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.989999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.990092 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:07.990340 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:07.990342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:07.990344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:07.990505 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:07.990638 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:07.990855 (+ 217us) server_negotiation.cc:299] Negotiation successful
0928 06:40:07.990958 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 06:40:07.992199 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 8913
I20260928 06:40:07.997605 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 9047
I20260928 06:40:08.002461 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 9181
I20260928 06:40:08.006906 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 8819
2026-09-28T06:40:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2029 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:08 dist-test-slave-gtr8 krb5kdc[9324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:08 dist-test-slave-gtr8 krb5kdc[9324](info): set up 2 sockets
Sep 28 06:40:08 dist-test-slave-gtr8 krb5kdc[9324](info): commencing operation
krb5kdc: starting...
W20260928 06:40:08.963025 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:09 dist-test-slave-gtr8 krb5kdc[9324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:09.077577 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:34247
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:33121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:34247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:09.184387 9337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:09.184648 9337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:09.184687 9337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:09.188656 9337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:09.188707 9337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:09.188726 9337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:09.188741 9337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:09.188755 9337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:09.193964 9337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:33121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:34247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:34247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:09.194984 9337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:09.195994 9337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:09.204325 9343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:09.204312 9337 server_base.cc:1034] running on GCE node
W20260928 06:40:09.204221 9342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:09.204243 9345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:09.205111 9337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:09.206198 9337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:09.207351 9337 hybrid_clock.cc:648] HybridClock initialized: now 1790577609207326 us; error 39 us; skew 500 ppm
Sep 28 06:40:09 dist-test-slave-gtr8 krb5kdc[9324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:09.216686 9337 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:09.217667 9337 webserver.cc:460] Webserver started at http://127.8.151.254:39223/ using document root <none> and password file <none>
I20260928 06:40:09.218219 9337 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:09.218286 9337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:09.218497 9337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:09.220649 9337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "176c5c0eaddd452291dbd5dee2280ed4"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "38d01e572114461d080a62f8a1f703de"
server_key_iv: "1784157492cc135cbfa670fa7eecf316"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.221190 9337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "176c5c0eaddd452291dbd5dee2280ed4"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "38d01e572114461d080a62f8a1f703de"
server_key_iv: "1784157492cc135cbfa670fa7eecf316"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.225064 9337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:40:09.227460 9352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:09.228719 9337 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:40:09.228839 9337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "176c5c0eaddd452291dbd5dee2280ed4"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "38d01e572114461d080a62f8a1f703de"
server_key_iv: "1784157492cc135cbfa670fa7eecf316"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.228940 9337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:09.245182 9337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:09.248126 9337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:09.248291 9337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:09.255009 9337 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:34247
I20260928 06:40:09.255040 9404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:34247 every 8 connection(s)
I20260928 06:40:09.255913 9337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:09.258531 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:09.263446 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 9337
I20260928 06:40:09.263540 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:09.263760 8799 external_mini_cluster.cc:1428] Setting key 12fa347d0b3e6c37222048d28bdd29f4
I20260928 06:40:09.264087 9405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4: Bootstrap starting.
I20260928 06:40:09.266300 9405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:09.267076 9405 log.cc:824] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:09.269017 9405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4: No bootstrap required, opened a new log
Sep 28 06:40:09 dist-test-slave-gtr8 krb5kdc[9324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:09.273242 9405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176c5c0eaddd452291dbd5dee2280ed4" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34247 } }
I20260928 06:40:09.273519 9405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:09.273610 9405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176c5c0eaddd452291dbd5dee2280ed4, State: Initialized, Role: FOLLOWER
I20260928 06:40:09.274121 9405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176c5c0eaddd452291dbd5dee2280ed4" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34247 } }
I20260928 06:40:09.274309 9405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:09.274377 9405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:09.274477 9405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:09.275555 9405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176c5c0eaddd452291dbd5dee2280ed4" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34247 } }
I20260928 06:40:09.275890 9405 leader_election.cc:302] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 176c5c0eaddd452291dbd5dee2280ed4; no voters:
I20260928 06:40:09.276233 9405 leader_election.cc:288] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:09.276315 9410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:09.276530 9410 raft_consensus.cc:686] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [term 1 LEADER]: Becoming Leader. State: Replica: 176c5c0eaddd452291dbd5dee2280ed4, State: Running, Role: LEADER
I20260928 06:40:09.276882 9410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176c5c0eaddd452291dbd5dee2280ed4" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34247 } }
I20260928 06:40:09.277556 9405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:09.278292 9412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 176c5c0eaddd452291dbd5dee2280ed4. Latest consensus state: current_term: 1 leader_uuid: "176c5c0eaddd452291dbd5dee2280ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176c5c0eaddd452291dbd5dee2280ed4" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34247 } } }
I20260928 06:40:09.278414 9412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:09.278748 9411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "176c5c0eaddd452291dbd5dee2280ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176c5c0eaddd452291dbd5dee2280ed4" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34247 } } }
I20260928 06:40:09.278774 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.265126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.8.151.254:34247)
0928 06:40:09.265548 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:09.265564 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:09.265598 (+ 34us) server_negotiation.cc:407] Connection header received
0928 06:40:09.266237 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:09.266253 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:09.266604 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:09.266877 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:09.267726 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.268708 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.269828 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.270318 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.270440 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.273086 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:09.273093 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:09.273108 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:09.273137 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:09.275038 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:09.275481 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:09.275487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:09.275493 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:09.275638 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:09.275944 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:09.275947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:09.275949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:09.276462 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:09.277283 (+ 821us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:09.277426 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 06:40:09.277731 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
I20260928 06:40:09.278838 9411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:09.279238 9419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:09.282001 9419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:09.287189 9419 catalog_manager.cc:1347] Generated new cluster ID: 418853b181964be0a6fac1e62bcead54
I20260928 06:40:09.287247 9419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:09.303766 9419 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:09.305059 9419 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:09.311851 9419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 176c5c0eaddd452291dbd5dee2280ed4: Generated new TSK 0
I20260928 06:40:09.312731 9419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:40:09.343982 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34247
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:33121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:09.449922 9431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:09.450157 9431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:09.450214 9431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:09.454090 9431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:09.454142 9431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:09.454236 9431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:09.459249 9431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:33121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34247
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:09.460214 9431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:09.461172 9431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:09.470135 9439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:09.470168 9437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:09.470130 9436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:09.470369 9431 server_base.cc:1034] running on GCE node
I20260928 06:40:09.470816 9431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:09.471400 9431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:09.472540 9431 hybrid_clock.cc:648] HybridClock initialized: now 1790577609472517 us; error 35 us; skew 500 ppm
Sep 28 06:40:09 dist-test-slave-gtr8 krb5kdc[9324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:09.475069 9431 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:40:09.475862 9431 webserver.cc:460] Webserver started at http://127.8.151.193:42105/ using document root <none> and password file <none>
I20260928 06:40:09.476341 9431 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:09.476394 9431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:09.476541 9431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:09.478518 9431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "208f643be7194bbe853c13293772eec5"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "c7526ad3e1ad312d98a7326f198a10de"
server_key_iv: "270301d6d56a35f104ef989cf7292971"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.479030 9431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "208f643be7194bbe853c13293772eec5"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "c7526ad3e1ad312d98a7326f198a10de"
server_key_iv: "270301d6d56a35f104ef989cf7292971"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.482481 9431 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:40:09.484687 9446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:09.485795 9431 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:09.485883 9431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "208f643be7194bbe853c13293772eec5"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "c7526ad3e1ad312d98a7326f198a10de"
server_key_iv: "270301d6d56a35f104ef989cf7292971"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.485976 9431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:09.505116 9431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:09.508109 9431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:09.508273 9431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:09.508826 9431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:09.509609 9431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:09.509670 9431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:09.509727 9431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:09.509755 9431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:09.517692 9431 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:46573
I20260928 06:40:09.517740 9559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:46573 every 8 connection(s)
I20260928 06:40:09.518640 9431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:09.519918 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 9431
I20260928 06:40:09.520027 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:09.520311 8799 external_mini_cluster.cc:1428] Setting key ed7840f9cb871b07b28d184533a03af4
Sep 28 06:40:09 dist-test-slave-gtr8 krb5kdc[9324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:40:09.532586 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.520328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:54485 (local address 127.8.151.254:34247)
0928 06:40:09.520445 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:09.520449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:09.521413 (+ 964us) server_negotiation.cc:407] Connection header received
0928 06:40:09.522710 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:09.522714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:09.522766 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:09.522853 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:09.524386 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.524898 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.526522 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.526748 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.526821 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.529352 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:09.529355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:09.529357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:09.529379 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:09.530535 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:09.530994 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:09.530997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:09.530999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:09.531090 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:09.531419 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:09.531423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:09.531424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:09.531620 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:09.531725 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:09.532320 (+ 595us) server_negotiation.cc:299] Negotiation successful
0928 06:40:09.532425 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 06:40:09.533560 9562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.520793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:34247 (local address 127.8.151.193:54485)
0928 06:40:09.521245 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:09.521299 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:09.522504 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:09.523054 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:09.523064 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:09.523579 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:09.524230 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.524248 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.525012 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.525015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.526390 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.526401 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.526888 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.526892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.527098 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.527664 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:09.527686 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:09.529187 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:09.530637 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:09.530642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:09.530673 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:09.530903 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:09.531201 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:09.531204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:09.531206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:09.531334 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:09.531782 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:09.531788 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:09.532054 (+ 266us) client_negotiation.cc:769] Sending connection context
0928 06:40:09.532269 (+ 215us) client_negotiation.cc:240] Negotiation successful
0928 06:40:09.532591 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":115,"threads_started":1}
I20260928 06:40:09.534835 9560 heartbeater.cc:347] Connected to a master server at 127.8.151.254:34247
I20260928 06:40:09.535066 9560 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:09.535560 9560 heartbeater.cc:511] Master 127.8.151.254:34247 requested a full tablet report, sending...
I20260928 06:40:09.536991 9369 ts_manager.cc:194] Registered new tserver with Master: 208f643be7194bbe853c13293772eec5 (127.8.151.193:46573)
I20260928 06:40:09.538065 9369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:54485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:40:09.545910 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34247
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:33121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:09.651929 9565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:09.652195 9565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:09.652256 9565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:09.656225 9565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:09.656281 9565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:09.656358 9565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:09.661527 9565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:33121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34247
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:09.662524 9565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:09.663530 9565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:09.672874 9573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:09.672892 9571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:09.672943 9565 server_base.cc:1034] running on GCE node
W20260928 06:40:09.672892 9570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:09.673445 9565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:09.674021 9565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:09.675200 9565 hybrid_clock.cc:648] HybridClock initialized: now 1790577609675181 us; error 32 us; skew 500 ppm
Sep 28 06:40:09 dist-test-slave-gtr8 krb5kdc[9324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:09.677874 9565 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:40:09.678810 9565 webserver.cc:460] Webserver started at http://127.8.151.194:35457/ using document root <none> and password file <none>
I20260928 06:40:09.679364 9565 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:09.679425 9565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:09.679590 9565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:09.681550 9565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "ec5da1a221d0409d90e02968620ef621"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "630ec70093329f16d57fff5c089f287a"
server_key_iv: "7a60008c14883bd30604246489ed594c"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.682029 9565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "ec5da1a221d0409d90e02968620ef621"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "630ec70093329f16d57fff5c089f287a"
server_key_iv: "7a60008c14883bd30604246489ed594c"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.685433 9565 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260928 06:40:09.687755 9580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:09.688944 9565 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:09.689051 9565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "ec5da1a221d0409d90e02968620ef621"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "630ec70093329f16d57fff5c089f287a"
server_key_iv: "7a60008c14883bd30604246489ed594c"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.689163 9565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:09.698129 9565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:09.700977 9565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:09.701138 9565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:09.701668 9565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:09.702412 9565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:09.702455 9565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:09.702491 9565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:09.702505 9565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:09.710453 9565 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:44915
I20260928 06:40:09.710476 9693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:44915 every 8 connection(s)
I20260928 06:40:09.711441 9565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
Sep 28 06:40:09 dist-test-slave-gtr8 krb5kdc[9324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:09.721320 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 9565
I20260928 06:40:09.721434 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:09.721788 8799 external_mini_cluster.cc:1428] Setting key 4924ed2ab918b53cff55d57622b50250
I20260928 06:40:09.724813 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.713047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:35931 (local address 127.8.151.254:34247)
0928 06:40:09.713186 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:09.713191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:09.714007 (+ 816us) server_negotiation.cc:407] Connection header received
0928 06:40:09.715399 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:09.715402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:09.715448 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:09.715526 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:09.716609 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.717123 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.718145 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.718351 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.718422 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.720989 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:09.720992 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:09.720994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:09.721017 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:09.722598 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:09.723194 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:09.723198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:09.723200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:09.723309 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:09.723711 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:09.723715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:09.723717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:09.723929 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:09.724027 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:09.724499 (+ 472us) server_negotiation.cc:299] Negotiation successful
0928 06:40:09.724654 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 06:40:09.725771 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.713454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:34247 (local address 127.8.151.194:35931)
0928 06:40:09.713865 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:09.713905 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:09.715201 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:09.715674 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:09.715680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:09.716003 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:09.716469 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.716481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.717211 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.717214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.718041 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.718049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.718458 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.718460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.718592 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.719203 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:09.719220 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:09.720842 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:09.722708 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:09.722713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:09.722727 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:09.723054 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:09.723410 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:09.723412 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:09.723414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:09.723566 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:09.724040 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:09.724045 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:09.724275 (+ 230us) client_negotiation.cc:769] Sending connection context
0928 06:40:09.724431 (+ 156us) client_negotiation.cc:240] Negotiation successful
0928 06:40:09.724730 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":114,"threads_started":1}
I20260928 06:40:09.726877 9694 heartbeater.cc:347] Connected to a master server at 127.8.151.254:34247
I20260928 06:40:09.727109 9694 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:09.727604 9694 heartbeater.cc:511] Master 127.8.151.254:34247 requested a full tablet report, sending...
I20260928 06:40:09.728590 9369 ts_manager.cc:194] Registered new tserver with Master: ec5da1a221d0409d90e02968620ef621 (127.8.151.194:44915)
I20260928 06:40:09.729313 9369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:35931
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:40:09.750046 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34247
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:33121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:09.859382 9699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:09.859634 9699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:09.859683 9699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:09.863580 9699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:09.863632 9699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:09.863694 9699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:09.868808 9699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:33121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34247
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:09.869822 9699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:09.870785 9699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:09.879910 9704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:09.879940 9705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:09.880079 9699 server_base.cc:1034] running on GCE node
W20260928 06:40:09.879917 9707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:09.880510 9699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:09.881088 9699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:09.882234 9699 hybrid_clock.cc:648] HybridClock initialized: now 1790577609882211 us; error 40 us; skew 500 ppm
Sep 28 06:40:09 dist-test-slave-gtr8 krb5kdc[9324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:09.884810 9699 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:40:09.885798 9699 webserver.cc:460] Webserver started at http://127.8.151.195:41793/ using document root <none> and password file <none>
I20260928 06:40:09.886329 9699 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:09.886392 9699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:09.886574 9699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:09.888558 9699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "3bd9430ff574408cb260d1e8f966164e"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "e054580b9bb2b583b4488f87545b0338"
server_key_iv: "c308264714ef71a3760dd6f49fe880e4"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.889132 9699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "3bd9430ff574408cb260d1e8f966164e"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "e054580b9bb2b583b4488f87545b0338"
server_key_iv: "c308264714ef71a3760dd6f49fe880e4"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.892741 9699 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:09.895015 9714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:09.896293 9699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:09.896401 9699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "3bd9430ff574408cb260d1e8f966164e"
format_stamp: "Formatted at 2026-09-28 06:40:09 on dist-test-slave-gtr8"
server_key: "e054580b9bb2b583b4488f87545b0338"
server_key_iv: "c308264714ef71a3760dd6f49fe880e4"
server_key_version: "encryptionkey@0"
I20260928 06:40:09.896487 9699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:09.922861 9699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:09.925872 9699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:09.926045 9699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:09.926636 9699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:09.927589 9699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:09.927635 9699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:09.927691 9699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:09.927721 9699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:09.936053 9699 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:45977
I20260928 06:40:09.936080 9827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:45977 every 8 connection(s)
I20260928 06:40:09.936986 9699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:09.945995 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 9699
I20260928 06:40:09.946125 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
Sep 28 06:40:09 dist-test-slave-gtr8 krb5kdc[9324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577609, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:09.946403 8799 external_mini_cluster.cc:1428] Setting key ca7e7221b1989fa99e62a5ad7e712912
I20260928 06:40:09.950060 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.938786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:51589 (local address 127.8.151.254:34247)
0928 06:40:09.938932 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:09.938936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:09.939754 (+ 818us) server_negotiation.cc:407] Connection header received
0928 06:40:09.941032 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:09.941035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:09.941089 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:09.941162 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:09.942338 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.942908 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.944143 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.944355 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.944440 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.946893 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:09.946897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:09.946899 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:09.946927 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:09.948396 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:09.948848 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:09.948851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:09.948852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:09.948925 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:09.949191 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:09.949193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:09.949194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:09.949359 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:09.949429 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:09.949852 (+ 423us) server_negotiation.cc:299] Negotiation successful
0928 06:40:09.949939 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 06:40:09.950872 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.939190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:34247 (local address 127.8.151.195:51589)
0928 06:40:09.939612 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:09.939654 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:09.940832 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:09.941340 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:09.941347 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:09.941688 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:09.942147 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.942179 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.943025 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.943028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.944027 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.944037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.944489 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.944493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.944655 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.945167 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:09.945183 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:09.946713 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:09.948496 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:09.948501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:09.948514 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:09.948770 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:09.948996 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:09.948998 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:09.949000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:09.949123 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:09.949428 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:09.949432 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:09.949671 (+ 239us) client_negotiation.cc:769] Sending connection context
0928 06:40:09.949821 (+ 150us) client_negotiation.cc:240] Negotiation successful
0928 06:40:09.950061 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":122,"threads_started":1}
I20260928 06:40:09.951898 9828 heartbeater.cc:347] Connected to a master server at 127.8.151.254:34247
I20260928 06:40:09.952109 9828 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:09.952556 9828 heartbeater.cc:511] Master 127.8.151.254:34247 requested a full tablet report, sending...
I20260928 06:40:09.953483 9369 ts_manager.cc:194] Registered new tserver with Master: 3bd9430ff574408cb260d1e8f966164e (127.8.151.195:45977)
I20260928 06:40:09.954011 9369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:51589
I20260928 06:40:09.960057 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:40:09.970232 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.963289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.8.151.254:34247)
0928 06:40:09.963404 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:09.963408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:09.963647 (+ 239us) server_negotiation.cc:407] Connection header received
0928 06:40:09.963737 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:09.963744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:09.963786 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:09.963845 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:09.964463 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.964990 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.965854 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.966060 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.966137 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.967447 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:09.967450 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:09.967452 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:09.967471 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:09.968816 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:09.969207 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:09.969209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:09.969210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:09.969280 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:09.969536 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:09.969538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:09.969539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:09.969742 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:09.969839 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:09.970012 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 06:40:09.970109 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 06:40:09.975193 9369 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:09.978235 9369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:09.992818 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.987495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59956 (local address 127.8.151.193:46573)
0928 06:40:09.987744 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:09.987760 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:09.987780 (+ 20us) server_negotiation.cc:407] Connection header received
0928 06:40:09.987912 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:09.987917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:09.988144 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:09.988644 (+ 500us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:09.988841 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.990071 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.991528 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.992125 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.992220 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.992369 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:09.992563 (+ 194us) server_negotiation.cc:299] Negotiation successful
0928 06:40:09.992657 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":69,"threads_started":1}
I20260928 06:40:09.992816 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.986914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:46573 (local address 127.0.0.1:59956)
0928 06:40:09.987605 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:09.987631 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:09.987756 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:09.988309 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:09.988315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:09.988351 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:09.988498 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.988505 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.990203 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.990206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.991415 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.991422 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.992232 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.992235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.992333 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.992367 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 06:40:09.992532 (+ 165us) client_negotiation.cc:240] Negotiation successful
0928 06:40:09.992658 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":54,"spinlock_wait_cycles":6272,"thread_start_us":92,"threads_started":1}
I20260928 06:40:09.993021 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.986978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:44915 (local address 127.0.0.1:49184)
0928 06:40:09.987918 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:09.987930 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:09.988008 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:09.988566 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:09.988569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:09.988585 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:09.988680 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.988686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.989810 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.989813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.991383 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.991395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.992547 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.992551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.992716 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.992734 (+ 18us) client_negotiation.cc:769] Sending connection context
0928 06:40:09.992789 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 06:40:09.992877 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":84,"thread_start_us":73,"threads_started":1}
I20260928 06:40:09.993110 9840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.986844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41908 (local address 127.8.151.195:45977)
0928 06:40:09.988236 (+ 1392us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:09.988247 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:09.988437 (+ 190us) server_negotiation.cc:407] Connection header received
0928 06:40:09.988526 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:09.988531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:09.988706 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:09.988841 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:09.989016 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.990127 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.991401 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.992251 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.992355 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.992484 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:09.992871 (+ 387us) server_negotiation.cc:299] Negotiation successful
0928 06:40:09.992964 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1293,"thread_start_us":108,"threads_started":1}
I20260928 06:40:09.993415 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.987500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.8.151.194:44915)
0928 06:40:09.988049 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:09.988060 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:09.988081 (+ 21us) server_negotiation.cc:407] Connection header received
0928 06:40:09.988151 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:09.988156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:09.988344 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:09.988492 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:09.988787 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.989684 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.991513 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.992270 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.992354 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.992495 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:09.993195 (+ 700us) server_negotiation.cc:299] Negotiation successful
0928 06:40:09.993289 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":141,"threads_started":1}
I20260928 06:40:09.994263 9839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:09.986705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:45977 (local address 127.0.0.1:41908)
0928 06:40:09.987352 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:09.987403 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:09.988359 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:09.988822 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:09.988825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:09.988843 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:09.988924 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.988930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.990260 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.990263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.991304 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:09.991315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:09.992547 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:09.992551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:09.992689 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:09.992724 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 06:40:09.992789 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 06:40:09.994127 (+ 1338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":1257,"spinlock_wait_cycles":3712,"thread_start_us":70,"threads_started":1}
I20260928 06:40:09.996059 9628 tablet_service.cc:1467] Processing CreateTablet for tablet e8c6edc47f254c0d95c0a900cf0731de (DEFAULT_TABLE table=test-table [id=e47e479955a04e31afe4415fe63907d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:09.997021 9628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c6edc47f254c0d95c0a900cf0731de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:09.996840 9762 tablet_service.cc:1467] Processing CreateTablet for tablet e8c6edc47f254c0d95c0a900cf0731de (DEFAULT_TABLE table=test-table [id=e47e479955a04e31afe4415fe63907d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:09.996835 9494 tablet_service.cc:1467] Processing CreateTablet for tablet e8c6edc47f254c0d95c0a900cf0731de (DEFAULT_TABLE table=test-table [id=e47e479955a04e31afe4415fe63907d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:09.997848 9494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c6edc47f254c0d95c0a900cf0731de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:09.997848 9762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c6edc47f254c0d95c0a900cf0731de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:10.004788 9846 tablet_bootstrap.cc:492] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e: Bootstrap starting.
I20260928 06:40:10.005338 9845 tablet_bootstrap.cc:492] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621: Bootstrap starting.
I20260928 06:40:10.005635 9847 tablet_bootstrap.cc:492] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5: Bootstrap starting.
I20260928 06:40:10.007126 9846 tablet_bootstrap.cc:654] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:10.007540 9847 tablet_bootstrap.cc:654] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:10.007966 9845 tablet_bootstrap.cc:654] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:10.007942 9846 log.cc:824] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:10.008819 9845 log.cc:824] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:10.009776 9847 log.cc:824] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:10.010205 9846 tablet_bootstrap.cc:492] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e: No bootstrap required, opened a new log
I20260928 06:40:10.010444 9846 ts_tablet_manager.cc:1397] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260928 06:40:10.010943 9845 tablet_bootstrap.cc:492] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621: No bootstrap required, opened a new log
I20260928 06:40:10.011117 9845 ts_tablet_manager.cc:1397] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 06:40:10.011433 9847 tablet_bootstrap.cc:492] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5: No bootstrap required, opened a new log
I20260928 06:40:10.011559 9847 ts_tablet_manager.cc:1397] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 06:40:10.014276 9847 raft_consensus.cc:348] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd9430ff574408cb260d1e8f966164e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45977 } } peers { permanent_uuid: "208f643be7194bbe853c13293772eec5" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 46573 } } peers { permanent_uuid: "ec5da1a221d0409d90e02968620ef621" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44915 } }
I20260928 06:40:10.014489 9847 raft_consensus.cc:374] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:10.014519 9847 raft_consensus.cc:729] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 208f643be7194bbe853c13293772eec5, State: Initialized, Role: FOLLOWER
I20260928 06:40:10.014340 9846 raft_consensus.cc:348] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd9430ff574408cb260d1e8f966164e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45977 } } peers { permanent_uuid: "208f643be7194bbe853c13293772eec5" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 46573 } } peers { permanent_uuid: "ec5da1a221d0409d90e02968620ef621" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44915 } }
I20260928 06:40:10.014604 9846 raft_consensus.cc:374] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:10.014657 9846 raft_consensus.cc:729] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd9430ff574408cb260d1e8f966164e, State: Initialized, Role: FOLLOWER
I20260928 06:40:10.014802 9845 raft_consensus.cc:348] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd9430ff574408cb260d1e8f966164e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45977 } } peers { permanent_uuid: "208f643be7194bbe853c13293772eec5" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 46573 } } peers { permanent_uuid: "ec5da1a221d0409d90e02968620ef621" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44915 } }
I20260928 06:40:10.014930 9847 consensus_queue.cc:260] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd9430ff574408cb260d1e8f966164e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45977 } } peers { permanent_uuid: "208f643be7194bbe853c13293772eec5" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 46573 } } peers { permanent_uuid: "ec5da1a221d0409d90e02968620ef621" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44915 } }
I20260928 06:40:10.015041 9845 raft_consensus.cc:374] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:10.015121 9845 raft_consensus.cc:729] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5da1a221d0409d90e02968620ef621, State: Initialized, Role: FOLLOWER
I20260928 06:40:10.015110 9846 consensus_queue.cc:260] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd9430ff574408cb260d1e8f966164e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45977 } } peers { permanent_uuid: "208f643be7194bbe853c13293772eec5" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 46573 } } peers { permanent_uuid: "ec5da1a221d0409d90e02968620ef621" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44915 } }
I20260928 06:40:10.015702 9560 heartbeater.cc:503] Master 127.8.151.254:34247 was elected leader, sending a full tablet report...
I20260928 06:40:10.016074 9846 ts_tablet_manager.cc:1428] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:40:10.016238 9847 ts_tablet_manager.cc:1428] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260928 06:40:10.016372 9828 heartbeater.cc:503] Master 127.8.151.254:34247 was elected leader, sending a full tablet report...
I20260928 06:40:10.016590 9845 consensus_queue.cc:260] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd9430ff574408cb260d1e8f966164e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45977 } } peers { permanent_uuid: "208f643be7194bbe853c13293772eec5" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 46573 } } peers { permanent_uuid: "ec5da1a221d0409d90e02968620ef621" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44915 } }
I20260928 06:40:10.017611 9845 ts_tablet_manager.cc:1428] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:40:10.018312 9694 heartbeater.cc:503] Master 127.8.151.254:34247 was elected leader, sending a full tablet report...
W20260928 06:40:10.019699 9561 tablet.cc:2369] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:10.143497 9851 raft_consensus.cc:482] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:10.143705 9851 raft_consensus.cc:504] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd9430ff574408cb260d1e8f966164e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45977 } } peers { permanent_uuid: "208f643be7194bbe853c13293772eec5" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 46573 } } peers { permanent_uuid: "ec5da1a221d0409d90e02968620ef621" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44915 } }
I20260928 06:40:10.144881 9851 leader_election.cc:288] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bd9430ff574408cb260d1e8f966164e (127.8.151.195:45977), ec5da1a221d0409d90e02968620ef621 (127.8.151.194:44915)
I20260928 06:40:10.150007 9840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:10.145214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:38611 (local address 127.8.151.195:45977)
0928 06:40:10.145354 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:10.145359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:10.145559 (+ 200us) server_negotiation.cc:407] Connection header received
0928 06:40:10.145707 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:10.145710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:10.145765 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:10.145847 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:10.146232 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.147007 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.148723 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.149334 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.149458 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:10.149510 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:10.149804 (+ 294us) server_negotiation.cc:299] Negotiation successful
0928 06:40:10.149873 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 06:40:10.150007 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:10.145265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:33965 (local address 127.8.151.194:44915)
0928 06:40:10.145393 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:10.145398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:10.145596 (+ 198us) server_negotiation.cc:407] Connection header received
0928 06:40:10.145706 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:10.145710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:10.145765 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:10.145848 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:10.146233 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.147001 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.148722 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.149340 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.149458 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:10.149510 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:10.149804 (+ 294us) server_negotiation.cc:299] Negotiation successful
0928 06:40:10.149873 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:40:10.150081 9855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:10.145147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:44915 (local address 127.8.151.193:33965)
0928 06:40:10.145511 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:10.145522 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:10.145607 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:10.145875 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:10.145878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:10.145920 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:10.146137 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:10.146143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.147133 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.147161 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:10.148601 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:10.148610 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.149496 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.149501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:10.149643 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:10.149709 (+ 66us) client_negotiation.cc:769] Sending connection context
0928 06:40:10.149785 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 06:40:10.149886 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":198,"spinlock_wait_cycles":30464,"thread_start_us":118,"threads_started":1}
I20260928 06:40:10.150080 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:10.145122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:45977 (local address 127.8.151.193:38611)
0928 06:40:10.145461 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:10.145475 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:10.145595 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:10.145875 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:10.145878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:10.145922 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:10.146129 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:10.146137 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.147133 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.147161 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:10.148598 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:10.148610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.149517 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.149521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:10.149658 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:10.149703 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 06:40:10.149785 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 06:40:10.149888 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":154,"spinlock_wait_cycles":9472,"thread_start_us":101,"threads_started":1}
I20260928 06:40:10.150785 9782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c6edc47f254c0d95c0a900cf0731de" candidate_uuid: "208f643be7194bbe853c13293772eec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd9430ff574408cb260d1e8f966164e" is_pre_election: true
I20260928 06:40:10.150792 9648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c6edc47f254c0d95c0a900cf0731de" candidate_uuid: "208f643be7194bbe853c13293772eec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5da1a221d0409d90e02968620ef621" is_pre_election: true
I20260928 06:40:10.151062 9782 raft_consensus.cc:2454] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 208f643be7194bbe853c13293772eec5 in term 0.
I20260928 06:40:10.151069 9648 raft_consensus.cc:2454] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 208f643be7194bbe853c13293772eec5 in term 0.
I20260928 06:40:10.151573 9448 leader_election.cc:302] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 208f643be7194bbe853c13293772eec5, 3bd9430ff574408cb260d1e8f966164e, ec5da1a221d0409d90e02968620ef621; no voters:
I20260928 06:40:10.151847 9851 raft_consensus.cc:2781] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:10.151921 9851 raft_consensus.cc:482] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:10.151952 9851 raft_consensus.cc:3037] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:10.153009 9851 raft_consensus.cc:504] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd9430ff574408cb260d1e8f966164e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45977 } } peers { permanent_uuid: "208f643be7194bbe853c13293772eec5" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 46573 } } peers { permanent_uuid: "ec5da1a221d0409d90e02968620ef621" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44915 } }
I20260928 06:40:10.153363 9851 leader_election.cc:288] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [CANDIDATE]: Term 1 election: Requested vote from peers 3bd9430ff574408cb260d1e8f966164e (127.8.151.195:45977), ec5da1a221d0409d90e02968620ef621 (127.8.151.194:44915)
I20260928 06:40:10.153717 9782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c6edc47f254c0d95c0a900cf0731de" candidate_uuid: "208f643be7194bbe853c13293772eec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd9430ff574408cb260d1e8f966164e"
I20260928 06:40:10.153717 9648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c6edc47f254c0d95c0a900cf0731de" candidate_uuid: "208f643be7194bbe853c13293772eec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5da1a221d0409d90e02968620ef621"
I20260928 06:40:10.153838 9782 raft_consensus.cc:3037] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:10.153838 9648 raft_consensus.cc:3037] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:10.155102 9648 raft_consensus.cc:2454] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 208f643be7194bbe853c13293772eec5 in term 1.
I20260928 06:40:10.155100 9782 raft_consensus.cc:2454] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 208f643be7194bbe853c13293772eec5 in term 1.
I20260928 06:40:10.155439 9450 leader_election.cc:302] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 208f643be7194bbe853c13293772eec5, 3bd9430ff574408cb260d1e8f966164e; no voters:
I20260928 06:40:10.155591 9851 raft_consensus.cc:2781] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:10.155817 9851 raft_consensus.cc:686] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [term 1 LEADER]: Becoming Leader. State: Replica: 208f643be7194bbe853c13293772eec5, State: Running, Role: LEADER
I20260928 06:40:10.156181 9851 consensus_queue.cc:237] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd9430ff574408cb260d1e8f966164e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45977 } } peers { permanent_uuid: "208f643be7194bbe853c13293772eec5" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 46573 } } peers { permanent_uuid: "ec5da1a221d0409d90e02968620ef621" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44915 } }
I20260928 06:40:10.159214 9367 catalog_manager.cc:5495] T e8c6edc47f254c0d95c0a900cf0731de P 208f643be7194bbe853c13293772eec5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 208f643be7194bbe853c13293772eec5 (127.8.151.193). New cstate: current_term: 1 leader_uuid: "208f643be7194bbe853c13293772eec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd9430ff574408cb260d1e8f966164e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "208f643be7194bbe853c13293772eec5" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 46573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec5da1a221d0409d90e02968620ef621" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44915 } health_report { overall_health: UNKNOWN } } }
Sep 28 06:40:10 dist-test-slave-gtr8 krb5kdc[9324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:40:10.184458 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:10.175837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59966 (local address 127.8.151.193:46573)
0928 06:40:10.175982 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:10.175987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:10.176125 (+ 138us) server_negotiation.cc:407] Connection header received
0928 06:40:10.176263 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:10.176266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:10.176315 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:10.176379 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:10.177067 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.177619 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.178478 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.178764 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.178854 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:10.181270 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:10.181286 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:10.181301 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:10.181326 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:10.182783 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:10.183222 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:10.183230 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:10.183231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:10.183325 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:10.183568 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:10.183571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:10.183573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:10.183824 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:10.184025 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:10.184187 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 06:40:10.184299 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260928 06:40:10.187877 9829 tablet.cc:2369] T e8c6edc47f254c0d95c0a900cf0731de P 3bd9430ff574408cb260d1e8f966164e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 06:40:10 dist-test-slave-gtr8 krb5kdc[9324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 06:40:10 dist-test-slave-gtr8 krb5kdc[9324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:40:10.209991 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:10.202124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.8.151.193:46573)
0928 06:40:10.202252 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:10.202255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:10.202439 (+ 184us) server_negotiation.cc:407] Connection header received
0928 06:40:10.202533 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:10.202536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:10.202587 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:10.202693 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:10.203359 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.204140 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.205051 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.205284 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.205362 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:10.207536 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:10.207539 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:10.207541 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:10.207566 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:10.208693 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:10.209027 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:10.209029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:10.209031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:10.209095 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:10.209296 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:10.209298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:10.209299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:10.209445 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:10.209558 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:10.209748 (+ 190us) server_negotiation.cc:299] Negotiation successful
0928 06:40:10.209839 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260928 06:40:10.212549 9695 tablet.cc:2369] T e8c6edc47f254c0d95c0a900cf0731de P ec5da1a221d0409d90e02968620ef621: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 06:40:10 dist-test-slave-gtr8 krb5kdc[9324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 06:40:10 dist-test-slave-gtr8 krb5kdc[9324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:40:10.234957 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:10.226680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59974 (local address 127.8.151.193:46573)
0928 06:40:10.226822 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:10.226825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:10.226956 (+ 131us) server_negotiation.cc:407] Connection header received
0928 06:40:10.227066 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:10.227069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:10.227111 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:10.227256 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:10.227882 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.228386 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.229312 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:10.229521 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:10.229600 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:10.232059 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:10.232062 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:10.232064 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:10.232088 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:10.233312 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:10.233780 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:10.233783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:10.233785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:10.233863 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:10.234169 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:10.234172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:10.234174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:10.234370 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:10.234498 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:10.234714 (+ 216us) server_negotiation.cc:299] Negotiation successful
0928 06:40:10.234835 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260928 06:40:10.236342 9474 tablet_service.cc:2908] Rejecting scan request for tablet e8c6edc47f254c0d95c0a900cf0731de: Uninitialized: clean time has not yet been initialized
I20260928 06:40:10.238503 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 9431
I20260928 06:40:10.243605 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 9565
I20260928 06:40:10.248196 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 9699
I20260928 06:40:10.253099 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 9337
2026-09-28T06:40:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2247 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:10 dist-test-slave-gtr8 krb5kdc[9870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:10 dist-test-slave-gtr8 krb5kdc[9870](info): set up 2 sockets
Sep 28 06:40:10 dist-test-slave-gtr8 krb5kdc[9870](info): commencing operation
krb5kdc: starting...
W20260928 06:40:11.207440 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:11 dist-test-slave-gtr8 krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:11.322712 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:38525
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:38525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:11.430217 9883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:11.430474 9883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:11.430514 9883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:11.434582 9883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:11.434641 9883 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 06:40:11.434657 9883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:11.434675 9883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:11.434689 9883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:11.434702 9883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:11.440013 9883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:38525
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:38525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:11.440996 9883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:11.441946 9883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:11.450289 9888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:11.450371 9889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:11.450258 9891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:11.450481 9883 server_base.cc:1034] running on GCE node
I20260928 06:40:11.451045 9883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:11.452135 9883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:11.453341 9883 hybrid_clock.cc:648] HybridClock initialized: now 1790577611453307 us; error 36 us; skew 500 ppm
Sep 28 06:40:11 dist-test-slave-gtr8 krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577611, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:11.455979 9883 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:11.456817 9883 webserver.cc:460] Webserver started at http://127.8.151.254:44971/ using document root <none> and password file <none>
I20260928 06:40:11.457322 9883 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:11.457398 9883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:11.457584 9883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:11.459573 9883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "f6a876aaaecf4089b50aa06154660cab"
format_stamp: "Formatted at 2026-09-28 06:40:11 on dist-test-slave-gtr8"
server_key: "c73f37d23c84828a911ebee0fa060f94"
server_key_iv: "f59989f1c313499b90711bb362d25a00"
server_key_version: "encryptionkey@0"
I20260928 06:40:11.460075 9883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "f6a876aaaecf4089b50aa06154660cab"
format_stamp: "Formatted at 2026-09-28 06:40:11 on dist-test-slave-gtr8"
server_key: "c73f37d23c84828a911ebee0fa060f94"
server_key_iv: "f59989f1c313499b90711bb362d25a00"
server_key_version: "encryptionkey@0"
I20260928 06:40:11.463637 9883 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 06:40:11.465946 9898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:11.467315 9883 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:11.467433 9883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "f6a876aaaecf4089b50aa06154660cab"
format_stamp: "Formatted at 2026-09-28 06:40:11 on dist-test-slave-gtr8"
server_key: "c73f37d23c84828a911ebee0fa060f94"
server_key_iv: "f59989f1c313499b90711bb362d25a00"
server_key_version: "encryptionkey@0"
I20260928 06:40:11.467552 9883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:11.495407 9883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:11.500447 9883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:11.500681 9883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:11.509218 9883 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:38525
I20260928 06:40:11.509250 9960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:38525 every 8 connection(s)
I20260928 06:40:11.510177 9883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:11.512912 9961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:11.518194 9961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab: Bootstrap starting.
I20260928 06:40:11.518371 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 9883
I20260928 06:40:11.518467 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:11.518690 8799 external_mini_cluster.cc:1428] Setting key ed151df816aea8a0bb3494cad02c25be
I20260928 06:40:11.520596 9961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:11.521297 9961 log.cc:824] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:11.523252 9961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab: No bootstrap required, opened a new log
Sep 28 06:40:11 dist-test-slave-gtr8 krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:11.527287 9961 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a876aaaecf4089b50aa06154660cab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38525 } }
I20260928 06:40:11.527544 9961 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:11.527606 9961 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a876aaaecf4089b50aa06154660cab, State: Initialized, Role: FOLLOWER
I20260928 06:40:11.528069 9961 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a876aaaecf4089b50aa06154660cab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38525 } }
I20260928 06:40:11.528239 9961 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:11.528306 9961 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:11.528404 9961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:11.529448 9961 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a876aaaecf4089b50aa06154660cab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38525 } }
I20260928 06:40:11.529769 9961 leader_election.cc:302] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6a876aaaecf4089b50aa06154660cab; no voters:
I20260928 06:40:11.530087 9961 leader_election.cc:288] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:11.530162 9966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:11.530400 9966 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [term 1 LEADER]: Becoming Leader. State: Replica: f6a876aaaecf4089b50aa06154660cab, State: Running, Role: LEADER
I20260928 06:40:11.530750 9966 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a876aaaecf4089b50aa06154660cab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38525 } }
I20260928 06:40:11.531194 9961 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:11.532441 9967 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6a876aaaecf4089b50aa06154660cab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a876aaaecf4089b50aa06154660cab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38525 } } }
I20260928 06:40:11.532557 9968 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6a876aaaecf4089b50aa06154660cab. Latest consensus state: current_term: 1 leader_uuid: "f6a876aaaecf4089b50aa06154660cab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a876aaaecf4089b50aa06154660cab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38525 } } }
I20260928 06:40:11.532666 9968 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:11.533057 9978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:11.533252 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:11.519878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46126 (local address 127.8.151.254:38525)
0928 06:40:11.520258 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:11.520273 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:11.520308 (+ 35us) server_negotiation.cc:407] Connection header received
0928 06:40:11.520955 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:11.520973 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:11.521291 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:11.521570 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:11.522299 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:11.523332 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:11.524413 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:11.524861 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:11.524967 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:11.527498 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:11.527504 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:11.527520 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:11.527558 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:11.529433 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:11.529882 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:11.529887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:11.529893 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:11.530030 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:11.530335 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:11.530338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:11.530340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:11.530868 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:11.531048 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:11.531813 (+ 765us) server_negotiation.cc:299] Negotiation successful
0928 06:40:11.532100 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":107,"threads_started":1}
I20260928 06:40:11.532569 9967 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:11.536147 9978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:11.541846 9978 catalog_manager.cc:1347] Generated new cluster ID: 8807c856a3104ac2abc51b950632987c
I20260928 06:40:11.541896 9978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:11.555248 9978 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:11.556145 9978 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:11.561318 9978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab: Generated new TSK 0
I20260928 06:40:11.561965 9978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:40:11.592375 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38525
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:11.744792 9987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:11.745045 9987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:11.745090 9987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:11.749074 9987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:11.749140 9987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:11.749179 9987 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 06:40:11.749217 9987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:11.754426 9987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.151.254:38525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:11.755439 9987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:11.756412 9987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:11.765472 9995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:11.765455 9992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:11.765733 9987 server_base.cc:1034] running on GCE node
W20260928 06:40:11.765455 9993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:11.766109 9987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:11.766707 9987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:11.767865 9987 hybrid_clock.cc:648] HybridClock initialized: now 1790577611767835 us; error 39 us; skew 500 ppm
Sep 28 06:40:11 dist-test-slave-gtr8 krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577611, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:11.770488 9987 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:40:11.771409 9987 webserver.cc:460] Webserver started at http://127.8.151.193:40097/ using document root <none> and password file <none>
I20260928 06:40:11.771970 9987 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:11.772040 9987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:11.772235 9987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:11.774224 9987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "b8c2e60639204aeb8f3a4510db8a7623"
format_stamp: "Formatted at 2026-09-28 06:40:11 on dist-test-slave-gtr8"
server_key: "973e17102ad24cf65c295eaa6bc26079"
server_key_iv: "b4472e9d3cc104a87539060e3924f791"
server_key_version: "encryptionkey@0"
I20260928 06:40:11.774760 9987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "b8c2e60639204aeb8f3a4510db8a7623"
format_stamp: "Formatted at 2026-09-28 06:40:11 on dist-test-slave-gtr8"
server_key: "973e17102ad24cf65c295eaa6bc26079"
server_key_iv: "b4472e9d3cc104a87539060e3924f791"
server_key_version: "encryptionkey@0"
I20260928 06:40:11.778322 9987 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260928 06:40:11.780705 10002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:11.781956 9987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:11.782081 9987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "b8c2e60639204aeb8f3a4510db8a7623"
format_stamp: "Formatted at 2026-09-28 06:40:11 on dist-test-slave-gtr8"
server_key: "973e17102ad24cf65c295eaa6bc26079"
server_key_iv: "b4472e9d3cc104a87539060e3924f791"
server_key_version: "encryptionkey@0"
I20260928 06:40:11.782200 9987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:11.803493 9987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:11.806406 9987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:11.806583 9987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:11.808125 9987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:11.808175 9987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:11.808228 9987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:11.808252 9987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 06:40:11 dist-test-slave-gtr8 krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577611, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:11.818493 9987 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:32987
I20260928 06:40:11.819300 10117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:32987 every 8 connection(s)
I20260928 06:40:11.819561 9987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:11.820888 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 9987
I20260928 06:40:11.820988 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:11.821210 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:11.808865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:53647 (local address 127.8.151.254:38525)
0928 06:40:11.808980 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:11.808983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:11.810091 (+ 1108us) server_negotiation.cc:407] Connection header received
0928 06:40:11.811060 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:11.811064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:11.811124 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:11.811203 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:11.812509 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:11.813098 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:11.814529 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:11.814820 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:11.814922 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:11.817611 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:11.817614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:11.817616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:11.817639 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:11.819084 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:11.819683 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:11.819686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:11.819688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:11.819768 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:11.820162 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:11.820166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:11.820168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:11.820381 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:11.820466 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:11.820929 (+ 463us) server_negotiation.cc:299] Negotiation successful
0928 06:40:11.821050 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:40:11.821223 8799 external_mini_cluster.cc:1428] Setting key bd143d3a00f866dc7603748041e84a53
I20260928 06:40:11.822026 10011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:11.809388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38525 (local address 127.8.151.193:53647)
0928 06:40:11.809940 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:11.809984 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:11.810828 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:11.811549 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:11.811555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:11.811860 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:11.812370 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:11.812386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:11.813223 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:11.813226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:11.814397 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:11.814407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:11.814984 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:11.814989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:11.815220 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:11.815773 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:11.815793 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:11.817451 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:11.819243 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:11.819249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:11.819263 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:11.819569 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:11.819849 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:11.819852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:11.819854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:11.820040 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:11.820477 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:11.820483 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:11.820703 (+ 220us) client_negotiation.cc:769] Sending connection context
0928 06:40:11.820872 (+ 169us) client_negotiation.cc:240] Negotiation successful
0928 06:40:11.821165 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":195,"threads_started":1}
I20260928 06:40:11.823232 10118 heartbeater.cc:347] Connected to a master server at 127.8.151.254:38525
I20260928 06:40:11.823369 10118 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:11.823812 10118 heartbeater.cc:511] Master 127.8.151.254:38525 requested a full tablet report, sending...
I20260928 06:40:11.825176 9914 ts_manager.cc:194] Registered new tserver with Master: b8c2e60639204aeb8f3a4510db8a7623 (127.8.151.193:32987)
I20260928 06:40:11.826687 9914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:53647
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:40:11.833526 10125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:11.825346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38525 (local address 127.8.151.193:58531)
0928 06:40:11.825820 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:11.825831 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:11.825915 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:11.826128 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:11.826131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:11.826300 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:11.826389 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:11.826394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:11.827833 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:11.827837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:11.828681 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:11.828691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:11.829394 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:11.829398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:11.829496 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:11.829942 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:11.829955 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:11.830380 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:11.832017 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:11.832020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:11.832022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:11.832318 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:11.832556 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:11.832559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:11.832561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:11.832636 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:11.832972 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:11.832975 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:11.833058 (+ 83us) client_negotiation.cc:769] Sending connection context
0928 06:40:11.833139 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 06:40:11.833316 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":102,"threads_started":1}
I20260928 06:40:11.833541 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:11.825419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:58531 (local address 127.8.151.254:38525)
0928 06:40:11.825525 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:11.825528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:11.825897 (+ 369us) server_negotiation.cc:407] Connection header received
0928 06:40:11.825983 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:11.825986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:11.826026 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:11.826110 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:11.826487 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:11.827252 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:11.828818 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:11.829150 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:11.829269 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:11.830514 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:11.830518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:11.830548 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:11.830574 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:11.831920 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:11.832399 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:11.832403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:11.832405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:11.832480 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:11.832710 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:11.832713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:11.832715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:11.832888 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:11.832969 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:11.833227 (+ 258us) server_negotiation.cc:299] Negotiation successful
0928 06:40:11.833366 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":38}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:40:11.850055 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38525
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:11.957902 10127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:11.958179 10127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:11.958222 10127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:11.962277 10127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:11.962342 10127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:11.962381 10127 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 06:40:11.962436 10127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:11.967876 10127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.151.254:38525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:11.968938 10127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:11.969939 10127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:11.979512 10132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:11.979557 10133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:11.979555 10127 server_base.cc:1034] running on GCE node
W20260928 06:40:11.979512 10135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:11.980089 10127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:11.980731 10127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:11.981882 10127 hybrid_clock.cc:648] HybridClock initialized: now 1790577611981864 us; error 31 us; skew 500 ppm
Sep 28 06:40:11 dist-test-slave-gtr8 krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577611, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:11.984517 10127 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:40:11.985363 10127 webserver.cc:460] Webserver started at http://127.8.151.194:40051/ using document root <none> and password file <none>
I20260928 06:40:11.985875 10127 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:11.985936 10127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:11.986301 10127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:11.988380 10127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "7fb81dc1c6d04e48a2cd8e60def766a9"
format_stamp: "Formatted at 2026-09-28 06:40:11 on dist-test-slave-gtr8"
server_key: "6ad4282668d53691262ccbc8ab7dccb8"
server_key_iv: "c421eece122bb8c92ac8f8bce7af3b12"
server_key_version: "encryptionkey@0"
I20260928 06:40:11.988921 10127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "7fb81dc1c6d04e48a2cd8e60def766a9"
format_stamp: "Formatted at 2026-09-28 06:40:11 on dist-test-slave-gtr8"
server_key: "6ad4282668d53691262ccbc8ab7dccb8"
server_key_iv: "c421eece122bb8c92ac8f8bce7af3b12"
server_key_version: "encryptionkey@0"
I20260928 06:40:11.992673 10127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 06:40:11.995060 10142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:11.996315 10127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:11.996546 10127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "7fb81dc1c6d04e48a2cd8e60def766a9"
format_stamp: "Formatted at 2026-09-28 06:40:11 on dist-test-slave-gtr8"
server_key: "6ad4282668d53691262ccbc8ab7dccb8"
server_key_iv: "c421eece122bb8c92ac8f8bce7af3b12"
server_key_version: "encryptionkey@0"
I20260928 06:40:11.996654 10127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:12.013031 10127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:12.016489 10127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:12.016654 10127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:12.018049 10127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:12.018100 10127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:12.018152 10127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:12.018181 10127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 06:40:12 dist-test-slave-gtr8 krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577611, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:12.028424 10127 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:37451
I20260928 06:40:12.028468 10257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:37451 every 8 connection(s)
I20260928 06:40:12.029392 10127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:12.031567 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.018908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:58543 (local address 127.8.151.254:38525)
0928 06:40:12.019025 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.019028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.020201 (+ 1173us) server_negotiation.cc:407] Connection header received
0928 06:40:12.021259 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.021262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.021320 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.021397 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:12.022982 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.023899 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.025149 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.025371 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.025457 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.027948 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:12.027952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:12.027955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:12.027983 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:12.029486 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.030063 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.030067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.030069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.030156 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.030470 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.030473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.030475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.030658 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:12.030741 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.031238 (+ 497us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.031370 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 06:40:12.032447 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.019489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38525 (local address 127.8.151.194:58543)
0928 06:40:12.020040 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.020096 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.021035 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.021641 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.021651 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.022121 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:12.022827 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.022852 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.024030 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.024034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.025038 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.025047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.025506 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.025510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.025681 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.026295 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:12.026310 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:12.027776 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:12.029592 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:12.029597 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:12.029611 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:12.029942 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:12.030244 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:12.030247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:12.030250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:12.030382 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:12.030762 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:12.030768 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:12.030993 (+ 225us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.031185 (+ 192us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.031528 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":165,"threads_started":1}
I20260928 06:40:12.033535 10258 heartbeater.cc:347] Connected to a master server at 127.8.151.254:38525
I20260928 06:40:12.033670 10258 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:12.034246 10258 heartbeater.cc:511] Master 127.8.151.254:38525 requested a full tablet report, sending...
I20260928 06:40:12.035341 9915 ts_manager.cc:194] Registered new tserver with Master: 7fb81dc1c6d04e48a2cd8e60def766a9 (127.8.151.194:37451)
I20260928 06:40:12.036049 9915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:58543
I20260928 06:40:12.036326 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 10127
I20260928 06:40:12.036432 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:12.036717 8799 external_mini_cluster.cc:1428] Setting key 40fe020c42ff1cbb0c06e1e28157e692
I20260928 06:40:12.042933 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.035803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38525 (local address 127.8.151.194:55809)
0928 06:40:12.036041 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.036054 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.036147 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.036505 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.036508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.036723 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:12.036845 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.036853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.037652 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.037656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.038430 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.038439 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.038857 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.038860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.038945 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.039428 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:12.039439 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:12.039735 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:12.041527 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:12.041529 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:12.041531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:12.041743 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:12.041985 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:12.041987 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:12.041988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:12.042068 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:12.042515 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:12.042516 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:12.042588 (+ 72us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.042686 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.042797 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":90,"threads_started":1}
I20260928 06:40:12.043033 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.035858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:55809 (local address 127.8.151.254:38525)
0928 06:40:12.035963 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.035967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.036183 (+ 216us) server_negotiation.cc:407] Connection header received
0928 06:40:12.036231 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.036234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.036277 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.036351 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:12.036950 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.037526 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.038535 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.038761 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.038844 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.039857 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:12.039860 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:12.039862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:12.039883 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:12.041416 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.041826 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.041829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.041831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.041913 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.042154 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.042157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.042159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.042434 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:12.042534 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.042756 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.042876 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:40:12.063663 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38525
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:12.174139 10267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:12.174415 10267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:12.174463 10267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:12.178457 10267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:12.178517 10267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:12.178555 10267 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 06:40:12.178598 10267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:12.184070 10267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.151.254:38525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:12.185102 10267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:12.186143 10267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:12.195498 10272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:12.195617 10267 server_base.cc:1034] running on GCE node
W20260928 06:40:12.195497 10275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:12.195489 10273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:12.196208 10267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:12.196848 10267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:12.198021 10267 hybrid_clock.cc:648] HybridClock initialized: now 1790577612197990 us; error 43 us; skew 500 ppm
Sep 28 06:40:12 dist-test-slave-gtr8 krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577612, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:12.200758 10267 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:40:12.201591 10267 webserver.cc:460] Webserver started at http://127.8.151.195:40817/ using document root <none> and password file <none>
I20260928 06:40:12.202081 10267 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:12.202134 10267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:12.202307 10267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:12.204366 10267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "a4210d002528443da52e04844a2c3a9e"
format_stamp: "Formatted at 2026-09-28 06:40:12 on dist-test-slave-gtr8"
server_key: "61f25f6724e339e73484af5ccc00eebf"
server_key_iv: "b2db141c2919af7426944bd5be5c72e5"
server_key_version: "encryptionkey@0"
I20260928 06:40:12.204861 10267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "a4210d002528443da52e04844a2c3a9e"
format_stamp: "Formatted at 2026-09-28 06:40:12 on dist-test-slave-gtr8"
server_key: "61f25f6724e339e73484af5ccc00eebf"
server_key_iv: "b2db141c2919af7426944bd5be5c72e5"
server_key_version: "encryptionkey@0"
I20260928 06:40:12.208547 10267 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 06:40:12.210775 10282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:12.211977 10267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:12.212069 10267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "a4210d002528443da52e04844a2c3a9e"
format_stamp: "Formatted at 2026-09-28 06:40:12 on dist-test-slave-gtr8"
server_key: "61f25f6724e339e73484af5ccc00eebf"
server_key_iv: "b2db141c2919af7426944bd5be5c72e5"
server_key_version: "encryptionkey@0"
I20260928 06:40:12.212184 10267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:12.227787 10267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:12.230881 10267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:12.231068 10267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:12.232682 10267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:12.232729 10267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:12.232771 10267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:12.232800 10267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 06:40:12 dist-test-slave-gtr8 krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577612, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:12.242933 10267 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:43113
I20260928 06:40:12.242985 10397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:43113 every 8 connection(s)
I20260928 06:40:12.243887 10267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:12.246572 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.233408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:56935 (local address 127.8.151.254:38525)
0928 06:40:12.233526 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.233530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.234779 (+ 1249us) server_negotiation.cc:407] Connection header received
0928 06:40:12.235973 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.235978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.236037 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.236093 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:12.237789 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.238599 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.239960 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.240270 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.240386 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.243042 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:12.243046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:12.243049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:12.243081 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:12.244586 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.245134 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.245138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.245139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.245223 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.245525 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.245528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.245530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.245723 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:12.245807 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.246288 (+ 481us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.246409 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 06:40:12.247406 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.234122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38525 (local address 127.8.151.195:56935)
0928 06:40:12.234633 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.234677 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.235723 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.236487 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.236494 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.236926 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:12.237624 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.237645 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.238717 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.238719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.239863 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.239872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.240451 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.240454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.240603 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.241256 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:12.241274 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:12.242879 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:12.244703 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:12.244708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:12.244723 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:12.245040 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:12.245305 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:12.245308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:12.245310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:12.245442 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:12.245819 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:12.245826 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:12.246057 (+ 231us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.246226 (+ 169us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.246541 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":133,"threads_started":1}
I20260928 06:40:12.248569 10398 heartbeater.cc:347] Connected to a master server at 127.8.151.254:38525
I20260928 06:40:12.248709 10398 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:12.249269 10398 heartbeater.cc:511] Master 127.8.151.254:38525 requested a full tablet report, sending...
I20260928 06:40:12.249809 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 10267
I20260928 06:40:12.249930 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:12.250241 8799 external_mini_cluster.cc:1428] Setting key 4bd8754d0ec913cd1eae8576e62ac495
I20260928 06:40:12.250383 9914 ts_manager.cc:194] Registered new tserver with Master: a4210d002528443da52e04844a2c3a9e (127.8.151.195:43113)
I20260928 06:40:12.251036 9914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:56935
I20260928 06:40:12.252178 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:40:12.257721 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.251056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:59763 (local address 127.8.151.254:38525)
0928 06:40:12.251187 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.251191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.251204 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:12.251290 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.251293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.251340 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.251408 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:12.251994 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.252709 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.253724 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.253932 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.254017 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.255023 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:12.255027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:12.255029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:12.255053 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:12.256331 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.256687 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.256690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.256692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.256766 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.256995 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.256998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.257000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.257162 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:12.257250 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.257475 (+ 225us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.257572 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 06:40:12.257743 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.250794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38525 (local address 127.8.151.195:59763)
0928 06:40:12.251070 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.251085 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.251200 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.251517 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.251520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.251753 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:12.251876 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.251884 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.252848 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.252851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.253605 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.253616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.254034 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.254037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.254123 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.254583 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:12.254594 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:12.254897 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:12.256426 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:12.256429 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:12.256432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:12.256606 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:12.256842 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:12.256845 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:12.256847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:12.256921 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:12.257244 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:12.257247 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:12.257335 (+ 88us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.257416 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.257555 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
Sep 28 06:40:12 dist-test-slave-gtr8 krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 06:40:12 dist-test-slave-gtr8 krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:12.275811 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.268587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46138 (local address 127.8.151.254:38525)
0928 06:40:12.268696 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.268699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.268711 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:12.268752 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.268754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.268793 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.268861 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:12.269482 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.269983 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.270811 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.271016 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.271091 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.273356 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:12.273359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:12.273362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:12.273385 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:12.274519 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.274870 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.274872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.274874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.274941 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.275120 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.275122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.275123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.275329 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:12.275428 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.275600 (+ 172us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.275689 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 06:40:12.278434 9914 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:12.281466 9914 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:12.295526 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.289661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:37451 (local address 127.0.0.1:59516)
0928 06:40:12.290081 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.290126 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.290257 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.291050 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.291056 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.291077 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:12.291254 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.291262 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.292826 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.292829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.294052 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.294063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.294888 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.294891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.295087 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.295106 (+ 19us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.295246 (+ 140us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.295383 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":107,"thread_start_us":80,"threads_started":1}
I20260928 06:40:12.295536 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.289783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59516 (local address 127.8.151.194:37451)
0928 06:40:12.290539 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.290551 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.290573 (+ 22us) server_negotiation.cc:407] Connection header received
0928 06:40:12.290646 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.290652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.290883 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.291406 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:12.291528 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.292720 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.294183 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.294773 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.294873 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.295030 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.295279 (+ 249us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.295377 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":101,"threads_started":1}
I20260928 06:40:12.295608 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.290501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.8.151.193:32987)
0928 06:40:12.291242 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.291257 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.291280 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:40:12.291351 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.291356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.291533 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.291673 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:12.291831 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.292894 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.294206 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.294763 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.294852 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.295027 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.295362 (+ 335us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.295459 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":81,"threads_started":1}
I20260928 06:40:12.295948 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.290330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:32987 (local address 127.0.0.1:48684)
0928 06:40:12.290921 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.290932 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.291003 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.291649 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.291652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.291667 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:12.291746 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.291752 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.292991 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.292994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.294105 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.294116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.294878 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.294881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.295043 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.295105 (+ 62us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.295242 (+ 137us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.295821 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":564,"thread_start_us":75,"threads_started":1}
I20260928 06:40:12.296375 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.289778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:43113 (local address 127.0.0.1:38948)
0928 06:40:12.290321 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.290333 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.290402 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.291489 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.291492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.291508 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:12.291604 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.291610 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.292584 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.292587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.294071 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.294077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.296005 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.296008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.296108 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.296130 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.296182 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.296259 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":88,"thread_start_us":94,"threads_started":1}
I20260928 06:40:12.296468 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.290697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38948 (local address 127.8.151.195:43113)
0928 06:40:12.291085 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.291097 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.291118 (+ 21us) server_negotiation.cc:407] Connection header received
0928 06:40:12.291213 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.291218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.291372 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.291499 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:12.291721 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.292482 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.295097 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.295882 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.295989 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.296149 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.296224 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.296314 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":100,"threads_started":1}
I20260928 06:40:12.299206 10192 tablet_service.cc:1467] Processing CreateTablet for tablet 086d260be59045fd9126fb1839e8dab7 (DEFAULT_TABLE table=test-table [id=7f64971cf09c4a1eb79b2a652d555fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:12.299211 10052 tablet_service.cc:1467] Processing CreateTablet for tablet 086d260be59045fd9126fb1839e8dab7 (DEFAULT_TABLE table=test-table [id=7f64971cf09c4a1eb79b2a652d555fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:12.299540 10332 tablet_service.cc:1467] Processing CreateTablet for tablet 086d260be59045fd9126fb1839e8dab7 (DEFAULT_TABLE table=test-table [id=7f64971cf09c4a1eb79b2a652d555fc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:12.300390 10192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 086d260be59045fd9126fb1839e8dab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:12.300390 10052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 086d260be59045fd9126fb1839e8dab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:12.300446 10332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 086d260be59045fd9126fb1839e8dab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:12.307503 10421 tablet_bootstrap.cc:492] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: Bootstrap starting.
I20260928 06:40:12.309090 10423 tablet_bootstrap.cc:492] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: Bootstrap starting.
I20260928 06:40:12.309698 10421 tablet_bootstrap.cc:654] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:12.310403 10421 log.cc:824] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:12.311170 10423 tablet_bootstrap.cc:654] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:12.311468 10422 tablet_bootstrap.cc:492] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: Bootstrap starting.
I20260928 06:40:12.312011 10423 log.cc:824] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:12.312784 10421 tablet_bootstrap.cc:492] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: No bootstrap required, opened a new log
I20260928 06:40:12.312979 10421 ts_tablet_manager.cc:1397] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 06:40:12.313447 10422 tablet_bootstrap.cc:654] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:12.314072 10423 tablet_bootstrap.cc:492] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: No bootstrap required, opened a new log
I20260928 06:40:12.314074 10422 log.cc:824] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:12.314260 10423 ts_tablet_manager.cc:1397] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260928 06:40:12.315790 10422 tablet_bootstrap.cc:492] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: No bootstrap required, opened a new log
I20260928 06:40:12.315958 10422 ts_tablet_manager.cc:1397] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260928 06:40:12.316861 10421 raft_consensus.cc:348] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } }
I20260928 06:40:12.317121 10421 raft_consensus.cc:374] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:12.317211 10421 raft_consensus.cc:729] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4210d002528443da52e04844a2c3a9e, State: Initialized, Role: FOLLOWER
I20260928 06:40:12.317667 10421 consensus_queue.cc:260] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } }
I20260928 06:40:12.318094 10423 raft_consensus.cc:348] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } }
I20260928 06:40:12.318344 10423 raft_consensus.cc:374] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:12.318346 10398 heartbeater.cc:503] Master 127.8.151.254:38525 was elected leader, sending a full tablet report...
I20260928 06:40:12.318403 10423 raft_consensus.cc:729] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c2e60639204aeb8f3a4510db8a7623, State: Initialized, Role: FOLLOWER
I20260928 06:40:12.318804 10423 consensus_queue.cc:260] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } }
I20260928 06:40:12.319170 10422 raft_consensus.cc:348] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } }
I20260928 06:40:12.319832 10423 ts_tablet_manager.cc:1428] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260928 06:40:12.320030 10421 ts_tablet_manager.cc:1428] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260928 06:40:12.320215 10118 heartbeater.cc:503] Master 127.8.151.254:38525 was elected leader, sending a full tablet report...
W20260928 06:40:12.320617 10119 tablet.cc:2369] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:12.321132 10422 raft_consensus.cc:374] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:12.321208 10422 raft_consensus.cc:729] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb81dc1c6d04e48a2cd8e60def766a9, State: Initialized, Role: FOLLOWER
I20260928 06:40:12.321736 10422 consensus_queue.cc:260] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } }
I20260928 06:40:12.322333 10258 heartbeater.cc:503] Master 127.8.151.254:38525 was elected leader, sending a full tablet report...
I20260928 06:40:12.322674 10422 ts_tablet_manager.cc:1428] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: Time spent starting tablet: real 0.007s user 0.001s sys 0.004s
I20260928 06:40:12.363058 10429 raft_consensus.cc:482] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:12.363303 10429 raft_consensus.cc:504] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } }
I20260928 06:40:12.364415 10429 leader_election.cc:288] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4210d002528443da52e04844a2c3a9e (127.8.151.195:43113), b8c2e60639204aeb8f3a4510db8a7623 (127.8.151.193:32987)
I20260928 06:40:12.368522 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.364645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:43113 (local address 127.8.151.194:59095)
0928 06:40:12.364761 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.364776 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.364870 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.365179 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.365182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.365202 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:12.365321 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.365328 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.366316 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.366318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.367543 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.367554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.368195 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.368198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.368294 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.368314 (+ 20us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.368367 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.368420 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 06:40:12.368587 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.364747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:59095 (local address 127.8.151.195:43113)
0928 06:40:12.364860 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.364865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.364884 (+ 19us) server_negotiation.cc:407] Connection header received
0928 06:40:12.364966 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.364968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.365016 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.365106 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:12.365412 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.366196 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.367647 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.368088 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.368203 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.368255 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.368391 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.368456 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 06:40:12.369160 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.364920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:32987 (local address 127.8.151.194:41269)
0928 06:40:12.365219 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.365243 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.365328 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.365553 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.365556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.365570 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:12.365661 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.365665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.366517 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.366520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.368016 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.368024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.368798 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.368802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.368892 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.368914 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.368975 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.369037 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":98,"threads_started":1}
I20260928 06:40:12.369105 10352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "086d260be59045fd9126fb1839e8dab7" candidate_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4210d002528443da52e04844a2c3a9e" is_pre_election: true
I20260928 06:40:12.369179 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.365022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:41269 (local address 127.8.151.193:32987)
0928 06:40:12.365147 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.365151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.365311 (+ 160us) server_negotiation.cc:407] Connection header received
0928 06:40:12.365408 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.365411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.365457 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.365528 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:12.365745 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.366427 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.368127 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.368667 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.368777 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.368822 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.369010 (+ 188us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.369078 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 06:40:12.369412 10352 raft_consensus.cc:2454] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb81dc1c6d04e48a2cd8e60def766a9 in term 0.
I20260928 06:40:12.369676 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "086d260be59045fd9126fb1839e8dab7" candidate_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c2e60639204aeb8f3a4510db8a7623" is_pre_election: true
I20260928 06:40:12.369871 10144 leader_election.cc:302] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fb81dc1c6d04e48a2cd8e60def766a9, a4210d002528443da52e04844a2c3a9e; no voters:
I20260928 06:40:12.369915 10060 raft_consensus.cc:2454] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb81dc1c6d04e48a2cd8e60def766a9 in term 0.
I20260928 06:40:12.370160 10429 raft_consensus.cc:2781] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:12.370244 10429 raft_consensus.cc:482] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:12.370293 10429 raft_consensus.cc:3037] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:12.371382 10429 raft_consensus.cc:504] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } }
I20260928 06:40:12.371707 10429 leader_election.cc:288] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [CANDIDATE]: Term 1 election: Requested vote from peers a4210d002528443da52e04844a2c3a9e (127.8.151.195:43113), b8c2e60639204aeb8f3a4510db8a7623 (127.8.151.193:32987)
I20260928 06:40:12.372016 10352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "086d260be59045fd9126fb1839e8dab7" candidate_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4210d002528443da52e04844a2c3a9e"
I20260928 06:40:12.372083 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "086d260be59045fd9126fb1839e8dab7" candidate_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c2e60639204aeb8f3a4510db8a7623"
I20260928 06:40:12.372124 10352 raft_consensus.cc:3037] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:12.372195 10060 raft_consensus.cc:3037] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:12.373178 10060 raft_consensus.cc:2454] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb81dc1c6d04e48a2cd8e60def766a9 in term 1.
I20260928 06:40:12.373327 10352 raft_consensus.cc:2454] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb81dc1c6d04e48a2cd8e60def766a9 in term 1.
I20260928 06:40:12.373440 10144 leader_election.cc:302] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fb81dc1c6d04e48a2cd8e60def766a9, b8c2e60639204aeb8f3a4510db8a7623; no voters:
I20260928 06:40:12.373581 10429 raft_consensus.cc:2781] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:12.373791 10429 raft_consensus.cc:686] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 LEADER]: Becoming Leader. State: Replica: 7fb81dc1c6d04e48a2cd8e60def766a9, State: Running, Role: LEADER
I20260928 06:40:12.374122 10429 consensus_queue.cc:237] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } }
I20260928 06:40:12.377238 9915 catalog_manager.cc:5495] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fb81dc1c6d04e48a2cd8e60def766a9 (127.8.151.194). New cstate: current_term: 1 leader_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } health_report { overall_health: UNKNOWN } } }
I20260928 06:40:12.399115 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.395550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59518 (local address 127.8.151.194:37451)
0928 06:40:12.395677 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.395682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.395727 (+ 45us) server_negotiation.cc:407] Connection header received
0928 06:40:12.395949 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.395952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.396002 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.396094 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:12.396358 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.396854 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.397713 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.397946 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.398031 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.398340 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:12.398732 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:12.398828 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.398918 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.398972 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:40:12.404824 10352 raft_consensus.cc:1261] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e [term 1 FOLLOWER]: Refusing update from remote peer 7fb81dc1c6d04e48a2cd8e60def766a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:12.404824 10060 raft_consensus.cc:1261] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623 [term 1 FOLLOWER]: Refusing update from remote peer 7fb81dc1c6d04e48a2cd8e60def766a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:12.405491 10431 consensus_queue.cc:1035] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:12.405705 10429 consensus_queue.cc:1035] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:12.413757 10436 mvcc.cc:205] Tried to move back new op lower bound from 7334205900402503680 to 7334205900286877696. Current Snapshot: MvccSnapshot[applied={T|T < 7334205900402503680}]
I20260928 06:40:12.425320 9913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:12.425482 9913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:12.427738 9913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab: Sending DeleteTablet for 3 replicas of tablet 086d260be59045fd9126fb1839e8dab7
I20260928 06:40:12.428498 10192 tablet_service.cc:1514] Processing DeleteTablet for tablet 086d260be59045fd9126fb1839e8dab7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:12 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:59516
I20260928 06:40:12.428500 10332 tablet_service.cc:1514] Processing DeleteTablet for tablet 086d260be59045fd9126fb1839e8dab7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:12 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:38948
I20260928 06:40:12.428527 10052 tablet_service.cc:1514] Processing DeleteTablet for tablet 086d260be59045fd9126fb1839e8dab7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:12 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48684
I20260928 06:40:12.429018 10446 tablet_replica.cc:330] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: stopping tablet replica
I20260928 06:40:12.429209 10444 tablet_replica.cc:330] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: stopping tablet replica
I20260928 06:40:12.429365 10446 raft_consensus.cc:2229] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:12.429462 10444 raft_consensus.cc:2229] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:12.429710 10446 raft_consensus.cc:2258] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:12.429785 10444 raft_consensus.cc:2258] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:12.429858 9913 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:12.430256 9913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:12.431267 10444 ts_tablet_manager.cc:1905] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:12.432268 10446 ts_tablet_manager.cc:1905] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:12.434841 10052 tablet_service.cc:1467] Processing CreateTablet for tablet 9cee9bc82edd445c90f96d88185e99b8 (DEFAULT_TABLE table=test-table [id=cb9ba13b61ce4266b01425313a976cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:12.435005 10192 tablet_service.cc:1467] Processing CreateTablet for tablet 9cee9bc82edd445c90f96d88185e99b8 (DEFAULT_TABLE table=test-table [id=cb9ba13b61ce4266b01425313a976cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:12.435115 10052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cee9bc82edd445c90f96d88185e99b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:12.435056 10332 tablet_service.cc:1467] Processing CreateTablet for tablet 9cee9bc82edd445c90f96d88185e99b8 (DEFAULT_TABLE table=test-table [id=cb9ba13b61ce4266b01425313a976cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:12.435253 10192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cee9bc82edd445c90f96d88185e99b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:12.437337 10423 tablet_bootstrap.cc:492] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623: Bootstrap starting.
I20260928 06:40:12.437580 10332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cee9bc82edd445c90f96d88185e99b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:12.438302 10423 tablet_bootstrap.cc:654] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:12.438576 10444 ts_tablet_manager.cc:1918] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:12.438663 10444 log.cc:1196] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/086d260be59045fd9126fb1839e8dab7
I20260928 06:40:12.438894 10422 tablet_bootstrap.cc:492] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: Bootstrap starting.
I20260928 06:40:12.438944 10444 ts_tablet_manager.cc:1939] T 086d260be59045fd9126fb1839e8dab7 P 7fb81dc1c6d04e48a2cd8e60def766a9: Deleting consensus metadata
I20260928 06:40:12.439642 10421 tablet_bootstrap.cc:492] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: Bootstrap starting.
I20260928 06:40:12.439805 10422 tablet_bootstrap.cc:654] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:12.440086 9902 catalog_manager.cc:4841] TS 7fb81dc1c6d04e48a2cd8e60def766a9 (127.8.151.194:37451): tablet 086d260be59045fd9126fb1839e8dab7 (table test-table [id=7f64971cf09c4a1eb79b2a652d555fc5]) successfully deleted
I20260928 06:40:12.440366 10445 tablet_replica.cc:330] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: stopping tablet replica
I20260928 06:40:12.440507 10421 tablet_bootstrap.cc:654] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:12.440685 10445 raft_consensus.cc:2229] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:12.440995 10445 raft_consensus.cc:2258] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:12.442456 10445 ts_tablet_manager.cc:1905] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:12.442495 10446 ts_tablet_manager.cc:1918] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:12.442579 10446 log.cc:1196] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/086d260be59045fd9126fb1839e8dab7
I20260928 06:40:12.442860 10446 ts_tablet_manager.cc:1939] T 086d260be59045fd9126fb1839e8dab7 P a4210d002528443da52e04844a2c3a9e: Deleting consensus metadata
I20260928 06:40:12.443639 10421 tablet_bootstrap.cc:492] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: No bootstrap required, opened a new log
I20260928 06:40:12.443737 10421 ts_tablet_manager.cc:1397] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260928 06:40:12.444170 10421 raft_consensus.cc:348] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.444276 10421 raft_consensus.cc:374] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:12.444303 10421 raft_consensus.cc:729] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4210d002528443da52e04844a2c3a9e, State: Initialized, Role: FOLLOWER
I20260928 06:40:12.444413 10421 consensus_queue.cc:260] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.444675 10421 ts_tablet_manager.cc:1428] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:12.444841 10445 ts_tablet_manager.cc:1918] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:12.444924 10445 log.cc:1196] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/086d260be59045fd9126fb1839e8dab7
I20260928 06:40:12.445013 9900 catalog_manager.cc:4841] TS a4210d002528443da52e04844a2c3a9e (127.8.151.195:43113): tablet 086d260be59045fd9126fb1839e8dab7 (table test-table [id=7f64971cf09c4a1eb79b2a652d555fc5]) successfully deleted
I20260928 06:40:12.445598 10445 ts_tablet_manager.cc:1939] T 086d260be59045fd9126fb1839e8dab7 P b8c2e60639204aeb8f3a4510db8a7623: Deleting consensus metadata
I20260928 06:40:12.446552 9900 catalog_manager.cc:4841] TS b8c2e60639204aeb8f3a4510db8a7623 (127.8.151.193:32987): tablet 086d260be59045fd9126fb1839e8dab7 (table test-table [id=7f64971cf09c4a1eb79b2a652d555fc5]) successfully deleted
I20260928 06:40:12.446913 10422 tablet_bootstrap.cc:492] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: No bootstrap required, opened a new log
I20260928 06:40:12.446996 10422 ts_tablet_manager.cc:1397] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260928 06:40:12.447496 10422 raft_consensus.cc:348] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.447608 10422 raft_consensus.cc:374] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:12.447640 10422 raft_consensus.cc:729] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb81dc1c6d04e48a2cd8e60def766a9, State: Initialized, Role: FOLLOWER
I20260928 06:40:12.447753 10422 consensus_queue.cc:260] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.447835 10423 tablet_bootstrap.cc:492] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623: No bootstrap required, opened a new log
I20260928 06:40:12.447901 10423 ts_tablet_manager.cc:1397] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260928 06:40:12.448004 10422 ts_tablet_manager.cc:1428] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260928 06:40:12.448299 10423 raft_consensus.cc:348] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.448401 10423 raft_consensus.cc:374] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:12.448433 10423 raft_consensus.cc:729] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c2e60639204aeb8f3a4510db8a7623, State: Initialized, Role: FOLLOWER
I20260928 06:40:12.448544 10423 consensus_queue.cc:260] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.448820 10423 ts_tablet_manager.cc:1428] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260928 06:40:12.494935 10399 tablet.cc:2369] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 06:40:12.530472 10259 tablet.cc:2369] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:12.628542 10439 raft_consensus.cc:482] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:12.628662 10439 raft_consensus.cc:504] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.629051 10439 leader_election.cc:288] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c2e60639204aeb8f3a4510db8a7623 (127.8.151.193:32987), a4210d002528443da52e04844a2c3a9e (127.8.151.195:43113)
I20260928 06:40:12.629539 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cee9bc82edd445c90f96d88185e99b8" candidate_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c2e60639204aeb8f3a4510db8a7623" is_pre_election: true
I20260928 06:40:12.629539 10352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cee9bc82edd445c90f96d88185e99b8" candidate_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4210d002528443da52e04844a2c3a9e" is_pre_election: true
I20260928 06:40:12.629696 10352 raft_consensus.cc:2454] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb81dc1c6d04e48a2cd8e60def766a9 in term 0.
I20260928 06:40:12.629698 10060 raft_consensus.cc:2454] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb81dc1c6d04e48a2cd8e60def766a9 in term 0.
I20260928 06:40:12.629964 10144 leader_election.cc:302] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fb81dc1c6d04e48a2cd8e60def766a9, a4210d002528443da52e04844a2c3a9e; no voters:
I20260928 06:40:12.630127 10439 raft_consensus.cc:2781] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:12.630185 10439 raft_consensus.cc:482] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:12.630249 10439 raft_consensus.cc:3037] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:12.631340 10439 raft_consensus.cc:504] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.631629 10439 leader_election.cc:288] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [CANDIDATE]: Term 1 election: Requested vote from peers b8c2e60639204aeb8f3a4510db8a7623 (127.8.151.193:32987), a4210d002528443da52e04844a2c3a9e (127.8.151.195:43113)
I20260928 06:40:12.631994 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cee9bc82edd445c90f96d88185e99b8" candidate_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c2e60639204aeb8f3a4510db8a7623"
I20260928 06:40:12.632048 10352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cee9bc82edd445c90f96d88185e99b8" candidate_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4210d002528443da52e04844a2c3a9e"
I20260928 06:40:12.632098 10060 raft_consensus.cc:3037] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:12.632165 10352 raft_consensus.cc:3037] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:12.633026 10060 raft_consensus.cc:2454] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb81dc1c6d04e48a2cd8e60def766a9 in term 1.
I20260928 06:40:12.633045 10352 raft_consensus.cc:2454] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb81dc1c6d04e48a2cd8e60def766a9 in term 1.
I20260928 06:40:12.633278 10144 leader_election.cc:302] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fb81dc1c6d04e48a2cd8e60def766a9, a4210d002528443da52e04844a2c3a9e; no voters:
I20260928 06:40:12.633430 10439 raft_consensus.cc:2781] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:12.633517 10439 raft_consensus.cc:686] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 LEADER]: Becoming Leader. State: Replica: 7fb81dc1c6d04e48a2cd8e60def766a9, State: Running, Role: LEADER
I20260928 06:40:12.633692 10439 consensus_queue.cc:237] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.635205 9915 catalog_manager.cc:5495] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fb81dc1c6d04e48a2cd8e60def766a9 (127.8.151.194). New cstate: current_term: 1 leader_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } health_report { overall_health: UNKNOWN } } }
Sep 28 06:40:12 dist-test-slave-gtr8 krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577611, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:12.705077 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.696977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38525 (local address 127.0.0.1:46146)
0928 06:40:12.697308 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.697322 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.697442 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.697695 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.697697 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.698117 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:12.698211 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.698217 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.698925 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.698929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.699519 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.699528 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.699950 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.699953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.700069 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.700644 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:12.700656 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:12.702054 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:12.703603 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:12.703613 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:12.703618 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:12.703847 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:12.704081 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:12.704084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:12.704086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:12.704158 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:12.704498 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:12.704506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:12.704619 (+ 113us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.704715 (+ 96us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.704874 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":17,"thread_start_us":124,"threads_started":1}
I20260928 06:40:12.705077 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.697116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46146 (local address 127.8.151.254:38525)
0928 06:40:12.697283 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.697287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.697393 (+ 106us) server_negotiation.cc:407] Connection header received
0928 06:40:12.697527 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.697530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.697570 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.697653 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:12.698324 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.698830 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.699646 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.699855 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.699937 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.702194 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:12.702196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:12.702198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:12.702219 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:12.703455 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.703935 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.703938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.703940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.704007 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.704231 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.704234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.704235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.704405 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:12.704489 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.704770 (+ 281us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.704922 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":76}
I20260928 06:40:12.708998 9915 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:46146:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 06:40:12.709743 9915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:12.714597 10052 tablet_service.cc:1467] Processing CreateTablet for tablet 2024f1d1e572407689c5b40ea7d42896 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddafb1d331284c949944a3b9d77dfcb6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 06:40:12.714597 10192 tablet_service.cc:1467] Processing CreateTablet for tablet 2024f1d1e572407689c5b40ea7d42896 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddafb1d331284c949944a3b9d77dfcb6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 06:40:12.714829 10332 tablet_service.cc:1467] Processing CreateTablet for tablet 2024f1d1e572407689c5b40ea7d42896 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddafb1d331284c949944a3b9d77dfcb6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 06:40:12.714998 10052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2024f1d1e572407689c5b40ea7d42896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:12.714998 10192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2024f1d1e572407689c5b40ea7d42896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:12.715231 10332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2024f1d1e572407689c5b40ea7d42896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:12.717690 10422 tablet_bootstrap.cc:492] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: Bootstrap starting.
I20260928 06:40:12.717690 10421 tablet_bootstrap.cc:492] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: Bootstrap starting.
I20260928 06:40:12.718125 10423 tablet_bootstrap.cc:492] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: Bootstrap starting.
I20260928 06:40:12.718544 10422 tablet_bootstrap.cc:654] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:12.718607 10421 tablet_bootstrap.cc:654] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:12.719020 10423 tablet_bootstrap.cc:654] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:12.720134 10423 tablet_bootstrap.cc:492] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: No bootstrap required, opened a new log
I20260928 06:40:12.720204 10422 tablet_bootstrap.cc:492] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: No bootstrap required, opened a new log
I20260928 06:40:12.720212 10423 ts_tablet_manager.cc:1397] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 06:40:12.720264 10421 tablet_bootstrap.cc:492] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: No bootstrap required, opened a new log
I20260928 06:40:12.720292 10422 ts_tablet_manager.cc:1397] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260928 06:40:12.720336 10421 ts_tablet_manager.cc:1397] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260928 06:40:12.720593 10423 raft_consensus.cc:348] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.720692 10423 raft_consensus.cc:374] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:12.720721 10423 raft_consensus.cc:729] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c2e60639204aeb8f3a4510db8a7623, State: Initialized, Role: FOLLOWER
I20260928 06:40:12.720705 10422 raft_consensus.cc:348] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.720705 10421 raft_consensus.cc:348] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.720821 10422 raft_consensus.cc:374] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:12.720821 10421 raft_consensus.cc:374] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:12.720855 10421 raft_consensus.cc:729] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4210d002528443da52e04844a2c3a9e, State: Initialized, Role: FOLLOWER
I20260928 06:40:12.720855 10422 raft_consensus.cc:729] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb81dc1c6d04e48a2cd8e60def766a9, State: Initialized, Role: FOLLOWER
I20260928 06:40:12.720853 10423 consensus_queue.cc:260] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.720971 10422 consensus_queue.cc:260] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.720971 10421 consensus_queue.cc:260] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.721112 10423 ts_tablet_manager.cc:1428] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 06:40:12.721225 10421 ts_tablet_manager.cc:1428] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:12.721225 10422 ts_tablet_manager.cc:1428] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260928 06:40:12.721465 10428 tablet_replica.cc:439] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } }
I20260928 06:40:12.721451 10427 tablet_replica.cc:439] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } }
I20260928 06:40:12.721596 10427 tablet_replica.cc:442] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:12.721606 10428 tablet_replica.cc:442] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:12.721776 10439 tablet_replica.cc:439] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } }
I20260928 06:40:12.721918 10439 tablet_replica.cc:442] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:12.791761 10427 raft_consensus.cc:482] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:12.791901 10427 raft_consensus.cc:504] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.793052 10427 leader_election.cc:288] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c2e60639204aeb8f3a4510db8a7623 (127.8.151.193:32987), 7fb81dc1c6d04e48a2cd8e60def766a9 (127.8.151.194:37451)
I20260928 06:40:12.797451 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.793269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:32987 (local address 127.8.151.195:54639)
0928 06:40:12.793577 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.793590 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.793688 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.793931 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.793933 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.793949 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:12.794084 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.794090 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.794815 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.794819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.796250 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.796260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.797016 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.797020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.797129 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.797169 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.797230 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.797287 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":57,"thread_start_us":83,"threads_started":1}
I20260928 06:40:12.797472 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.793333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:54639 (local address 127.8.151.193:32987)
0928 06:40:12.793455 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.793459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.793665 (+ 206us) server_negotiation.cc:407] Connection header received
0928 06:40:12.793785 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.793788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.793845 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.793904 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:12.794178 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.794693 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.796402 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.796901 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.797001 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.797054 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.797268 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.797350 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 06:40:12.797546 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.793406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:54671 (local address 127.8.151.194:37451)
0928 06:40:12.793559 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.793564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.793747 (+ 183us) server_negotiation.cc:407] Connection header received
0928 06:40:12.793861 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.793863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.793910 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.793978 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:12.794215 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.794698 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.796389 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.796917 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.797037 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.797090 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.797367 (+ 277us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.797445 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 06:40:12.797569 10460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.793317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:37451 (local address 127.8.151.195:54671)
0928 06:40:12.793646 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.793659 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.793768 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.793997 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.794000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.794015 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:12.794113 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.794117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.794791 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.794795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.796251 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.796260 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.797148 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.797151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.797233 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.797266 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.797336 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.797400 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":166,"thread_start_us":86,"threads_started":1}
I20260928 06:40:12.798149 10205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2024f1d1e572407689c5b40ea7d42896" candidate_uuid: "a4210d002528443da52e04844a2c3a9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" is_pre_election: true
I20260928 06:40:12.798381 10205 raft_consensus.cc:2454] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4210d002528443da52e04844a2c3a9e in term 0.
I20260928 06:40:12.798410 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2024f1d1e572407689c5b40ea7d42896" candidate_uuid: "a4210d002528443da52e04844a2c3a9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c2e60639204aeb8f3a4510db8a7623" is_pre_election: true
I20260928 06:40:12.798537 10060 raft_consensus.cc:2454] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4210d002528443da52e04844a2c3a9e in term 0.
I20260928 06:40:12.798795 10286 leader_election.cc:302] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7fb81dc1c6d04e48a2cd8e60def766a9, a4210d002528443da52e04844a2c3a9e, b8c2e60639204aeb8f3a4510db8a7623; no voters:
I20260928 06:40:12.799062 10427 raft_consensus.cc:2781] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:12.799118 10427 raft_consensus.cc:482] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:12.799182 10427 raft_consensus.cc:3037] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:12.800258 10427 raft_consensus.cc:504] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.800568 10427 leader_election.cc:288] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [CANDIDATE]: Term 1 election: Requested vote from peers b8c2e60639204aeb8f3a4510db8a7623 (127.8.151.193:32987), 7fb81dc1c6d04e48a2cd8e60def766a9 (127.8.151.194:37451)
I20260928 06:40:12.800927 10205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2024f1d1e572407689c5b40ea7d42896" candidate_uuid: "a4210d002528443da52e04844a2c3a9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9"
I20260928 06:40:12.800928 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2024f1d1e572407689c5b40ea7d42896" candidate_uuid: "a4210d002528443da52e04844a2c3a9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c2e60639204aeb8f3a4510db8a7623"
I20260928 06:40:12.801051 10060 raft_consensus.cc:3037] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:12.801051 10205 raft_consensus.cc:3037] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:12.802099 10205 raft_consensus.cc:2454] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4210d002528443da52e04844a2c3a9e in term 1.
I20260928 06:40:12.802099 10060 raft_consensus.cc:2454] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4210d002528443da52e04844a2c3a9e in term 1.
I20260928 06:40:12.802358 10286 leader_election.cc:302] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fb81dc1c6d04e48a2cd8e60def766a9, a4210d002528443da52e04844a2c3a9e; no voters:
I20260928 06:40:12.802518 10427 raft_consensus.cc:2781] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:12.802683 10427 raft_consensus.cc:686] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [term 1 LEADER]: Becoming Leader. State: Replica: a4210d002528443da52e04844a2c3a9e, State: Running, Role: LEADER
I20260928 06:40:12.802914 10427 consensus_queue.cc:237] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } }
I20260928 06:40:12.804061 10462 tablet_replica.cc:439] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: TxnStatusTablet state changed. Reason: New leader a4210d002528443da52e04844a2c3a9e. Latest consensus state: current_term: 1 leader_uuid: "a4210d002528443da52e04844a2c3a9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } }
I20260928 06:40:12.804183 10462 tablet_replica.cc:442] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: This TxnStatusTablet replica's current role is: LEADER
I20260928 06:40:12.804423 10464 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 06:40:12.804565 10464 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 06:40:12.805302 9913 catalog_manager.cc:5495] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e reported cstate change: term changed from 0 to 1, leader changed from <none> to a4210d002528443da52e04844a2c3a9e (127.8.151.195). New cstate: current_term: 1 leader_uuid: "a4210d002528443da52e04844a2c3a9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } health_report { overall_health: HEALTHY } } }
I20260928 06:40:12.825008 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.820464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38962 (local address 127.8.151.195:43113)
0928 06:40:12.820594 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.820598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.820684 (+ 86us) server_negotiation.cc:407] Connection header received
0928 06:40:12.820870 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.820873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.820935 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.821039 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:12.821303 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.822101 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.823342 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.823685 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.823809 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.824111 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:12.824619 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:12.824729 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.824807 (+ 78us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.824867 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 06:40:12.825042 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.820383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:43113 (local address 127.0.0.1:38962)
0928 06:40:12.820612 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.820626 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.820735 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.821053 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.821055 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.821064 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:12.821171 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.821185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.822219 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.822222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.823225 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.823236 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.823808 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.823811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.823906 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.823962 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:12.824712 (+ 750us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:12.824727 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.824795 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.824889 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260928 06:40:12.828204 10060 raft_consensus.cc:1261] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623 [term 1 FOLLOWER]: Refusing update from remote peer a4210d002528443da52e04844a2c3a9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:12.828588 10205 raft_consensus.cc:1261] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 FOLLOWER]: Refusing update from remote peer a4210d002528443da52e04844a2c3a9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:12.828668 10427 consensus_queue.cc:1035] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:12.829061 10427 consensus_queue.cc:1035] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:12.830602 10439 tablet_replica.cc:439] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: TxnStatusTablet state changed. Reason: New leader a4210d002528443da52e04844a2c3a9e. Latest consensus state: current_term: 1 leader_uuid: "a4210d002528443da52e04844a2c3a9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } }
I20260928 06:40:12.830713 10439 tablet_replica.cc:442] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:12.830590 10428 tablet_replica.cc:439] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: TxnStatusTablet state changed. Reason: New leader a4210d002528443da52e04844a2c3a9e. Latest consensus state: current_term: 1 leader_uuid: "a4210d002528443da52e04844a2c3a9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } }
I20260928 06:40:12.831012 10428 tablet_replica.cc:442] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:12.837255 10462 tablet_replica.cc:439] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a4210d002528443da52e04844a2c3a9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } }
I20260928 06:40:12.837385 10428 tablet_replica.cc:439] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a4210d002528443da52e04844a2c3a9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } }
I20260928 06:40:12.837495 10428 tablet_replica.cc:442] T 2024f1d1e572407689c5b40ea7d42896 P b8c2e60639204aeb8f3a4510db8a7623: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:12.837568 10462 tablet_replica.cc:442] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: This TxnStatusTablet replica's current role is: LEADER
I20260928 06:40:12.837553 10435 mvcc.cc:205] Tried to move back new op lower bound from 7334205902139326464 to 7334205902043136000. Current Snapshot: MvccSnapshot[applied={T|T < 7334205902139326464 or (T in {7334205902139326464})}]
I20260928 06:40:12.837832 10462 tablet_replica.cc:439] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a4210d002528443da52e04844a2c3a9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } }
I20260928 06:40:12.838577 10434 mvcc.cc:205] Tried to move back new op lower bound from 7334205902139326464 to 7334205902043136000. Current Snapshot: MvccSnapshot[applied={T|T < 7334205902139326464}]
I20260928 06:40:12.839080 10439 tablet_replica.cc:439] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a4210d002528443da52e04844a2c3a9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 } } peers { permanent_uuid: "7fb81dc1c6d04e48a2cd8e60def766a9" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 37451 } } peers { permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 } } }
I20260928 06:40:12.839205 10439 tablet_replica.cc:442] T 2024f1d1e572407689c5b40ea7d42896 P 7fb81dc1c6d04e48a2cd8e60def766a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:12.839978 10462 tablet_replica.cc:442] T 2024f1d1e572407689c5b40ea7d42896 P a4210d002528443da52e04844a2c3a9e: This TxnStatusTablet replica's current role is: LEADER
I20260928 06:40:12.852411 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.848853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:43113 (local address 127.8.151.194:50691)
0928 06:40:12.849247 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.849260 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.849346 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.849561 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.849563 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.849568 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:12.849664 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.849671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.850650 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.850652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.851307 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.851316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.851833 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.851836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.851892 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.851932 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:12.852180 (+ 248us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:12.852183 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.852225 (+ 42us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.852290 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":132,"threads_started":1}
I20260928 06:40:12.852412 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.848935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:50691 (local address 127.8.151.195:43113)
0928 06:40:12.849050 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.849055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.849321 (+ 266us) server_negotiation.cc:407] Connection header received
0928 06:40:12.849425 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.849428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.849477 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.849554 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:12.849753 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.850551 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.851398 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.851734 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.851825 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.852035 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:12.852113 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:12.852177 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.852237 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.852287 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 06:40:12.858520 10060 raft_consensus.cc:1261] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623 [term 1 FOLLOWER]: Refusing update from remote peer 7fb81dc1c6d04e48a2cd8e60def766a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:12.858549 10352 raft_consensus.cc:1261] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e [term 1 FOLLOWER]: Refusing update from remote peer 7fb81dc1c6d04e48a2cd8e60def766a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:12.858882 10431 consensus_queue.cc:1035] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c2e60639204aeb8f3a4510db8a7623" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 32987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:12.859030 10439 consensus_queue.cc:1035] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4210d002528443da52e04844a2c3a9e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:12.880731 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.876622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:37451 (local address 127.8.151.195:56981)
0928 06:40:12.877042 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:12.877055 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:12.877151 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:12.877380 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:12.877383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:12.877392 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:12.877676 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.877683 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.878377 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.878380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.879398 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:12.879407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.880063 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.880066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:12.880136 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.880177 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:12.880450 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:12.880456 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:40:12.880512 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 06:40:12.880587 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":96,"threads_started":1}
I20260928 06:40:12.880714 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.876687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:56981 (local address 127.8.151.194:37451)
0928 06:40:12.876787 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.876793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.877124 (+ 331us) server_negotiation.cc:407] Connection header received
0928 06:40:12.877259 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.877262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.877308 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.877365 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:12.877785 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.878283 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.879521 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.879832 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.879901 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.880309 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:12.880379 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:12.880434 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.880522 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.880569 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 06:40:12.904155 9915 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:12.904310 9915 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:12.906152 9915 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6a876aaaecf4089b50aa06154660cab: Sending DeleteTablet for 3 replicas of tablet 9cee9bc82edd445c90f96d88185e99b8
I20260928 06:40:12.906814 10192 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cee9bc82edd445c90f96d88185e99b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:12 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:59516
I20260928 06:40:12.906919 10052 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cee9bc82edd445c90f96d88185e99b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:12 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48684
I20260928 06:40:12.906917 10332 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cee9bc82edd445c90f96d88185e99b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:12 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:38948
I20260928 06:40:12.907097 10444 tablet_replica.cc:330] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: stopping tablet replica
I20260928 06:40:12.907096 10446 tablet_replica.cc:330] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: stopping tablet replica
I20260928 06:40:12.907225 10444 raft_consensus.cc:2229] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:12.907223 10446 raft_consensus.cc:2229] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:12.907331 10446 raft_consensus.cc:2258] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:12.907366 10444 raft_consensus.cc:2258] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:12.907485 10445 tablet_replica.cc:330] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623: stopping tablet replica
I20260928 06:40:12.907581 10445 raft_consensus.cc:2229] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:12.907680 10445 raft_consensus.cc:2258] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:12.908238 10444 ts_tablet_manager.cc:1905] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:12.908311 10446 ts_tablet_manager.cc:1905] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:12.910878 10444 ts_tablet_manager.cc:1918] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:12.910878 10446 ts_tablet_manager.cc:1918] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:12.910953 10444 log.cc:1196] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/9cee9bc82edd445c90f96d88185e99b8
I20260928 06:40:12.910953 10446 log.cc:1196] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/9cee9bc82edd445c90f96d88185e99b8
I20260928 06:40:12.911222 10444 ts_tablet_manager.cc:1939] T 9cee9bc82edd445c90f96d88185e99b8 P 7fb81dc1c6d04e48a2cd8e60def766a9: Deleting consensus metadata
I20260928 06:40:12.911238 10446 ts_tablet_manager.cc:1939] T 9cee9bc82edd445c90f96d88185e99b8 P a4210d002528443da52e04844a2c3a9e: Deleting consensus metadata
I20260928 06:40:12.911902 9902 catalog_manager.cc:4841] TS 7fb81dc1c6d04e48a2cd8e60def766a9 (127.8.151.194:37451): tablet 9cee9bc82edd445c90f96d88185e99b8 (table test-table [id=cb9ba13b61ce4266b01425313a976cf1]) successfully deleted
I20260928 06:40:12.912470 9900 catalog_manager.cc:4841] TS a4210d002528443da52e04844a2c3a9e (127.8.151.195:43113): tablet 9cee9bc82edd445c90f96d88185e99b8 (table test-table [id=cb9ba13b61ce4266b01425313a976cf1]) successfully deleted
I20260928 06:40:12.913031 10445 ts_tablet_manager.cc:1905] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:12.915740 10445 ts_tablet_manager.cc:1918] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:12.915818 10445 log.cc:1196] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/9cee9bc82edd445c90f96d88185e99b8
I20260928 06:40:12.916044 10445 ts_tablet_manager.cc:1939] T 9cee9bc82edd445c90f96d88185e99b8 P b8c2e60639204aeb8f3a4510db8a7623: Deleting consensus metadata
I20260928 06:40:12.916689 9900 catalog_manager.cc:4841] TS b8c2e60639204aeb8f3a4510db8a7623 (127.8.151.193:32987): tablet 9cee9bc82edd445c90f96d88185e99b8 (table test-table [id=cb9ba13b61ce4266b01425313a976cf1]) successfully deleted
Sep 28 06:40:12 dist-test-slave-gtr8 krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:40:12.921763 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.912595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48696 (local address 127.8.151.193:32987)
0928 06:40:12.912727 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.912732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.913132 (+ 400us) server_negotiation.cc:407] Connection header received
0928 06:40:12.913298 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.913302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.913347 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.913407 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:12.914044 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.914581 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.915502 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.915793 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.915887 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.918608 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:12.918617 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:12.918628 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:12.918652 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:12.920147 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.920640 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.920644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.920645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.920717 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.920937 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.920939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.920941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.921161 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:12.921317 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.921545 (+ 228us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.921639 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260928 06:40:12.922204 10112 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48696
I20260928 06:40:12.930778 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:12.923788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.8.151.254:38525)
0928 06:40:12.923910 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:12.923914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:12.923971 (+ 57us) server_negotiation.cc:407] Connection header received
0928 06:40:12.924102 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:12.924105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:12.924150 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:12.924212 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:12.924888 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.925389 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.926454 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:12.926650 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:12.926715 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:12.928120 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:12.928123 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:12.928125 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:12.928146 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:12.929355 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.929788 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.929791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.929792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.929859 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:12.930088 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:12.930090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:12.930091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:12.930258 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:12.930348 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:12.930562 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 06:40:12.930651 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260928 06:40:12.931217 9915 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46154
I20260928 06:40:12.932410 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 9987
I20260928 06:40:12.938616 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 10127
I20260928 06:40:12.943925 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 10267
I20260928 06:40:12.948992 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 9883
2026-09-28T06:40:12Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2695 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:13 dist-test-slave-gtr8 krb5kdc[10484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:13 dist-test-slave-gtr8 krb5kdc[10484](info): set up 2 sockets
Sep 28 06:40:13 dist-test-slave-gtr8 krb5kdc[10484](info): commencing operation
krb5kdc: starting...
W20260928 06:40:13.911242 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:13 dist-test-slave-gtr8 krb5kdc[10484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:14.023321 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:42257
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:42257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:14.129220 10497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:14.129487 10497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:14.129524 10497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:14.133603 10497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:14.133673 10497 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 06:40:14.133690 10497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:14.133706 10497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:14.133721 10497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:14.133735 10497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:14.139149 10497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:42257
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:42257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:14.140156 10497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:14.141109 10497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:14.149207 10505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:14.149237 10497 server_base.cc:1034] running on GCE node
W20260928 06:40:14.149207 10502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:14.149211 10503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:14.149972 10497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:14.151086 10497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:14.152279 10497 hybrid_clock.cc:648] HybridClock initialized: now 1790577614152264 us; error 28 us; skew 500 ppm
Sep 28 06:40:14 dist-test-slave-gtr8 krb5kdc[10484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577614, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:14.155035 10497 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:14.155941 10497 webserver.cc:460] Webserver started at http://127.8.151.254:37573/ using document root <none> and password file <none>
I20260928 06:40:14.156461 10497 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:14.156514 10497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:14.156678 10497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:14.158655 10497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "e570349406b34b0d99ca698b2eb6db07"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "64ee307d0f1f124a94a25be97131333e"
server_key_iv: "a630ba20f00e2309e5eab1e9a0ee780a"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.159194 10497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "e570349406b34b0d99ca698b2eb6db07"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "64ee307d0f1f124a94a25be97131333e"
server_key_iv: "a630ba20f00e2309e5eab1e9a0ee780a"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.162744 10497 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:40:14.165109 10512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:14.166316 10497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:14.166415 10497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "e570349406b34b0d99ca698b2eb6db07"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "64ee307d0f1f124a94a25be97131333e"
server_key_iv: "a630ba20f00e2309e5eab1e9a0ee780a"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.166507 10497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:14.196789 10497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:14.199872 10497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:14.200030 10497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:14.208145 10497 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:42257
I20260928 06:40:14.208161 10574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:42257 every 8 connection(s)
I20260928 06:40:14.209213 10497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:14.212046 10575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:14.217504 10575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07: Bootstrap starting.
I20260928 06:40:14.219338 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 10497
I20260928 06:40:14.219429 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:14.219686 8799 external_mini_cluster.cc:1428] Setting key 4ec41a5725353860be8871c35b1b1914
I20260928 06:40:14.220042 10575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:14.220923 10575 log.cc:824] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:14.222776 10575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07: No bootstrap required, opened a new log
I20260928 06:40:14.226812 10575 raft_consensus.cc:348] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570349406b34b0d99ca698b2eb6db07" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42257 } }
I20260928 06:40:14.227068 10575 raft_consensus.cc:374] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:14.227123 10575 raft_consensus.cc:729] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e570349406b34b0d99ca698b2eb6db07, State: Initialized, Role: FOLLOWER
I20260928 06:40:14.227557 10575 consensus_queue.cc:260] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570349406b34b0d99ca698b2eb6db07" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42257 } }
I20260928 06:40:14.227674 10575 raft_consensus.cc:388] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:14.227717 10575 raft_consensus.cc:482] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:14.227818 10575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:14.228650 10575 raft_consensus.cc:504] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570349406b34b0d99ca698b2eb6db07" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42257 } }
I20260928 06:40:14.228979 10575 leader_election.cc:302] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e570349406b34b0d99ca698b2eb6db07; no voters:
I20260928 06:40:14.229255 10575 leader_election.cc:288] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:14.229369 10580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:14.229601 10580 raft_consensus.cc:686] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [term 1 LEADER]: Becoming Leader. State: Replica: e570349406b34b0d99ca698b2eb6db07, State: Running, Role: LEADER
I20260928 06:40:14.229928 10580 consensus_queue.cc:237] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570349406b34b0d99ca698b2eb6db07" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42257 } }
I20260928 06:40:14.230394 10575 sys_catalog.cc:564] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:14.231480 10581 sys_catalog.cc:455] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e570349406b34b0d99ca698b2eb6db07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570349406b34b0d99ca698b2eb6db07" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42257 } } }
I20260928 06:40:14.231602 10581 sys_catalog.cc:458] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:14.231951 10589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:14.232990 10582 sys_catalog.cc:455] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e570349406b34b0d99ca698b2eb6db07. Latest consensus state: current_term: 1 leader_uuid: "e570349406b34b0d99ca698b2eb6db07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570349406b34b0d99ca698b2eb6db07" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42257 } } }
I20260928 06:40:14.233096 10582 sys_catalog.cc:458] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07 [sys.catalog]: This master's current role is: LEADER
Sep 28 06:40:14 dist-test-slave-gtr8 krb5kdc[10484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:14.235179 10589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:14.239269 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.220780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57304 (local address 127.8.151.254:42257)
0928 06:40:14.221251 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.221268 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.221303 (+ 35us) server_negotiation.cc:407] Connection header received
0928 06:40:14.221982 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.222000 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.222321 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.222590 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:14.223484 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.224583 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.225773 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.226262 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.226381 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.234039 (+ 7658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:14.234045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:14.234059 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:14.234088 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:14.235946 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.236386 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.236391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.236395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.236503 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.236759 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.236762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.236764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.237155 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:14.237322 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.238019 (+ 697us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.238319 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":156,"threads_started":1}
I20260928 06:40:14.241536 10589 catalog_manager.cc:1347] Generated new cluster ID: 4c3d9fda1d0c41c5a099b34bf70ddd7e
I20260928 06:40:14.241596 10589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:14.253043 10589 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:14.253947 10589 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:14.260602 10589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07: Generated new TSK 0
I20260928 06:40:14.261384 10589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:40:14.290660 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42257
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:14.398347 10601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:14.398614 10601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:14.398689 10601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:14.402704 10601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:14.402772 10601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:14.402827 10601 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 06:40:14.402854 10601 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 06:40:14.402904 10601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:14.408160 10601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.151.254:42257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:14.409222 10601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:14.410218 10601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:40:14.419261 10601 server_base.cc:1034] running on GCE node
W20260928 06:40:14.419190 10607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:14.419322 10606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:14.419322 10609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:14.419876 10601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:14.420478 10601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:14.421644 10601 hybrid_clock.cc:648] HybridClock initialized: now 1790577614421626 us; error 30 us; skew 500 ppm
Sep 28 06:40:14 dist-test-slave-gtr8 krb5kdc[10484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577614, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:14.424314 10601 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:40:14.425127 10601 webserver.cc:460] Webserver started at http://127.8.151.193:33523/ using document root <none> and password file <none>
I20260928 06:40:14.425639 10601 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:14.425704 10601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:14.425909 10601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:14.427999 10601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "5d21973ace6d4f4b9134cc426b07de88"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "15577aab6acccce512557d287e47a75b"
server_key_iv: "ff45b4063d2e5fa766e26d2788c18654"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.428617 10601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "5d21973ace6d4f4b9134cc426b07de88"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "15577aab6acccce512557d287e47a75b"
server_key_iv: "ff45b4063d2e5fa766e26d2788c18654"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.432380 10601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260928 06:40:14.434729 10616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:14.436036 10601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:14.436137 10601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "5d21973ace6d4f4b9134cc426b07de88"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "15577aab6acccce512557d287e47a75b"
server_key_iv: "ff45b4063d2e5fa766e26d2788c18654"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.436251 10601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:14.451561 10601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:14.454151 10601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:14.454324 10601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:14.455718 10601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:14.455771 10601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:14.455809 10601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:14.455837 10601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:14.465191 10601 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:45853
Sep 28 06:40:14 dist-test-slave-gtr8 krb5kdc[10484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577614, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:14.465289 10731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:45853 every 8 connection(s)
I20260928 06:40:14.466178 10601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:14.467020 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 10601
I20260928 06:40:14.467161 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:14.467417 8799 external_mini_cluster.cc:1428] Setting key 3f7d508140e6e6cf387f5702546d8d71
I20260928 06:40:14.469566 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.456389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:51723 (local address 127.8.151.254:42257)
0928 06:40:14.456546 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.456552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.457700 (+ 1148us) server_negotiation.cc:407] Connection header received
0928 06:40:14.458854 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.458858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.458913 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.459015 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:14.460480 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.461288 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.462775 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.463128 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.463269 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.465873 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:14.465877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:14.465880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:14.465909 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:14.467522 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.468055 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.468059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.468061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.468150 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.468503 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.468506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.468508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.468722 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:14.468818 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.469294 (+ 476us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.469411 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 06:40:14.470479 10625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.457126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42257 (local address 127.8.151.193:51723)
0928 06:40:14.457561 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.457609 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.458625 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.459186 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.459195 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.459688 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:14.460316 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.460336 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.461406 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.461410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.462641 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.462651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.463313 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.463318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.463519 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.464168 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:14.464189 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:14.465708 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:14.467640 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.467646 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.467659 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.467960 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.468263 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.468266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.468269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.468411 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.468816 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:14.468822 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:14.469055 (+ 233us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.469247 (+ 192us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.469560 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":112,"threads_started":1}
I20260928 06:40:14.471709 10732 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42257
I20260928 06:40:14.471939 10732 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:14.472517 10732 heartbeater.cc:511] Master 127.8.151.254:42257 requested a full tablet report, sending...
I20260928 06:40:14.474155 10529 ts_manager.cc:194] Registered new tserver with Master: 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853)
I20260928 06:40:14.475757 10529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:51723
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:40:14.481031 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.474022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:49597 (local address 127.8.151.254:42257)
0928 06:40:14.474129 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.474133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.474359 (+ 226us) server_negotiation.cc:407] Connection header received
0928 06:40:14.474464 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.474467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.474507 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.474570 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:14.474973 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.475523 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.476628 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.476961 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.477065 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.478150 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:14.478153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:14.478154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:14.478175 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:14.479537 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.479977 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.479980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.479982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.480062 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.480309 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.480316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.480318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.480479 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:14.480562 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.480789 (+ 227us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.480885 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 06:40:14.481047 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.473965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42257 (local address 127.8.151.193:49597)
0928 06:40:14.474281 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.474294 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.474396 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.474594 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.474597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.474780 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:14.474871 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.474876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.475637 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.475640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.476249 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.476258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.477075 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.477079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.477174 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.477606 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:14.477619 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:14.478032 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:14.479659 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.479662 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.479664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.479890 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.480141 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.480144 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.480146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.480219 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.480564 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:14.480567 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:14.480648 (+ 81us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.480729 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.480870 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:40:14.495241 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42257
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:14.600124 10741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:14.600360 10741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:14.600404 10741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:14.604249 10741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:14.604302 10741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:14.604339 10741 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 06:40:14.604352 10741 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 06:40:14.604382 10741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:14.609529 10741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.151.254:42257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:14.610558 10741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:14.611557 10741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:14.620568 10746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:14.620576 10747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:14.620576 10749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:14.620745 10741 server_base.cc:1034] running on GCE node
I20260928 06:40:14.621169 10741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:14.621731 10741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:14.622880 10741 hybrid_clock.cc:648] HybridClock initialized: now 1790577614622866 us; error 26 us; skew 500 ppm
Sep 28 06:40:14 dist-test-slave-gtr8 krb5kdc[10484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577614, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:14.625387 10741 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:40:14.626327 10741 webserver.cc:460] Webserver started at http://127.8.151.194:44319/ using document root <none> and password file <none>
I20260928 06:40:14.626837 10741 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:14.626900 10741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:14.627081 10741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:14.629099 10741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "e9cdbbc1620943d7b35e402f2427423f"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "290da6d8979db73bec3e9639046b4102"
server_key_iv: "8e725fdde0d8da761a0b28cfcf6192a6"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.629624 10741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "e9cdbbc1620943d7b35e402f2427423f"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "290da6d8979db73bec3e9639046b4102"
server_key_iv: "8e725fdde0d8da761a0b28cfcf6192a6"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.633291 10741 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:14.635736 10756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:14.636977 10741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:14.637085 10741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "e9cdbbc1620943d7b35e402f2427423f"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "290da6d8979db73bec3e9639046b4102"
server_key_iv: "8e725fdde0d8da761a0b28cfcf6192a6"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.637176 10741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:14.655524 10741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:14.658174 10741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:14.658349 10741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:14.659977 10741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:14.660027 10741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:14.660079 10741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:14.660102 10741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 06:40:14 dist-test-slave-gtr8 krb5kdc[10484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577614, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:14.669867 10741 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:42655
I20260928 06:40:14.670269 10871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:42655 every 8 connection(s)
I20260928 06:40:14.670800 10741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:14.672710 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.660708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:55163 (local address 127.8.151.254:42257)
0928 06:40:14.660861 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.660866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.661681 (+ 815us) server_negotiation.cc:407] Connection header received
0928 06:40:14.662722 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.662726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.662780 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.662842 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:14.664147 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.664764 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.665955 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.666211 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.666296 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.669065 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:14.669068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:14.669070 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:14.669095 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:14.670639 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.671250 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.671254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.671256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.671343 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.671715 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.671719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.671720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.671916 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:14.671993 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.672499 (+ 506us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.672594 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:40:14.673475 10765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.661071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42257 (local address 127.8.151.194:55163)
0928 06:40:14.661515 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.661571 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.662479 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.663198 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.663204 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.663521 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:14.664012 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.664027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.664885 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.664889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.665848 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.665857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.666371 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.666376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.666516 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.667076 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:14.667094 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:14.668891 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:14.670747 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.670753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.670767 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.671096 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.671444 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.671447 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.671449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.671615 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.672016 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:14.672021 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:14.672306 (+ 285us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.672445 (+ 139us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.672675 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":106,"threads_started":1}
I20260928 06:40:14.674574 10872 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42257
I20260928 06:40:14.674755 10872 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:14.675307 10872 heartbeater.cc:511] Master 127.8.151.254:42257 requested a full tablet report, sending...
I20260928 06:40:14.676414 10528 ts_manager.cc:194] Registered new tserver with Master: e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655)
I20260928 06:40:14.677089 10528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:55163
I20260928 06:40:14.680783 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 10741
I20260928 06:40:14.680918 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:14.681208 8799 external_mini_cluster.cc:1428] Setting key 03278cf2bdb79d11c614bc132e416b28
I20260928 06:40:14.683388 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.676823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:49115 (local address 127.8.151.254:42257)
0928 06:40:14.676933 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.676937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.677100 (+ 163us) server_negotiation.cc:407] Connection header received
0928 06:40:14.677176 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.677179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.677215 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.677287 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:14.677742 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.678238 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.679392 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.679616 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.679715 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.680593 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:14.680597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:14.680599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:14.680624 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:14.681930 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.682280 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.682284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.682286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.682376 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.682639 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.682642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.682643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.682838 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:14.682908 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.683188 (+ 280us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.683273 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 06:40:14.683440 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.676796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42257 (local address 127.8.151.194:49115)
0928 06:40:14.677031 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.677043 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.677118 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.677312 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.677315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.677498 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:14.677606 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.677613 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.678362 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.678365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.679268 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.679277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.679715 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.679718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.679796 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.680181 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:14.680191 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:14.680465 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:14.682016 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.682019 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.682020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.682184 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.682463 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.682466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.682468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.682558 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.682928 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:14.682931 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:14.683016 (+ 85us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.683107 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.683279 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:40:14.705775 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42257
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:14.812021 10881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:14.812296 10881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:14.812376 10881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:14.816442 10881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:14.816504 10881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:14.816560 10881 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 06:40:14.816584 10881 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 06:40:14.816634 10881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:14.822391 10881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.151.254:42257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:14.823534 10881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:14.824651 10881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:14.834146 10887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:14.834160 10886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:14.834174 10889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:14.834501 10881 server_base.cc:1034] running on GCE node
I20260928 06:40:14.834897 10881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:14.835533 10881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:14.836694 10881 hybrid_clock.cc:648] HybridClock initialized: now 1790577614836676 us; error 32 us; skew 500 ppm
Sep 28 06:40:14 dist-test-slave-gtr8 krb5kdc[10484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577614, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:14.839774 10881 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:40:14.840787 10881 webserver.cc:460] Webserver started at http://127.8.151.195:46837/ using document root <none> and password file <none>
I20260928 06:40:14.841336 10881 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:14.841394 10881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:14.841591 10881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:14.843627 10881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "10db4b9e0d7047f899f6e89c55b91e26"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "31d16ded534c9a708c97ba6bb34da0b9"
server_key_iv: "3074e80b4ecadb22b75d8637f693b020"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.844142 10881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "10db4b9e0d7047f899f6e89c55b91e26"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "31d16ded534c9a708c97ba6bb34da0b9"
server_key_iv: "3074e80b4ecadb22b75d8637f693b020"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.847648 10881 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:14.849849 10896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:14.851007 10881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:14.851106 10881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "10db4b9e0d7047f899f6e89c55b91e26"
format_stamp: "Formatted at 2026-09-28 06:40:14 on dist-test-slave-gtr8"
server_key: "31d16ded534c9a708c97ba6bb34da0b9"
server_key_iv: "3074e80b4ecadb22b75d8637f693b020"
server_key_version: "encryptionkey@0"
I20260928 06:40:14.851231 10881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:14.860630 10881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:14.863070 10881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:14.863270 10881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:14.864581 10881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:14.864626 10881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:14.864676 10881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:14.864706 10881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 06:40:14 dist-test-slave-gtr8 krb5kdc[10484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577614, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:14.874051 10881 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:43599
I20260928 06:40:14.874166 11011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:43599 every 8 connection(s)
I20260928 06:40:14.874939 10881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:14.877476 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.865281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:37037 (local address 127.8.151.254:42257)
0928 06:40:14.865421 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.865426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.866218 (+ 792us) server_negotiation.cc:407] Connection header received
0928 06:40:14.867264 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.867267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.867328 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.867388 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:14.868800 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.869634 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.871334 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.871595 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.871673 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.874383 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:14.874386 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:14.874388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:14.874410 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:14.875940 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.876350 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.876353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.876354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.876417 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.876663 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.876665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.876666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.876836 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:14.876904 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.877270 (+ 366us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.877359 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:40:14.878202 10905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.865645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42257 (local address 127.8.151.195:37037)
0928 06:40:14.866059 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.866106 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.867001 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.867769 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.867777 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.868139 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:14.868666 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.868678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.869763 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.869767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.871215 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.871226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.871729 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.871734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.871927 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.872532 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:14.872553 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:14.874189 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:14.876031 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.876035 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.876044 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.876278 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.876477 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.876479 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.876481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.876602 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.876906 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:14.876910 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:14.877104 (+ 194us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.877239 (+ 135us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.877477 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":107,"threads_started":1}
I20260928 06:40:14.879241 11012 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42257
I20260928 06:40:14.879379 11012 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:14.879805 11012 heartbeater.cc:511] Master 127.8.151.254:42257 requested a full tablet report, sending...
I20260928 06:40:14.880726 10529 ts_manager.cc:194] Registered new tserver with Master: 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599)
I20260928 06:40:14.881424 10529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:37037
I20260928 06:40:14.881814 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 10881
I20260928 06:40:14.881908 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:14.882144 8799 external_mini_cluster.cc:1428] Setting key 1bfb47c77966b05aa6bd904199678a93
I20260928 06:40:14.884089 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:40:14.888583 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.881186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:59959 (local address 127.8.151.254:42257)
0928 06:40:14.882443 (+ 1257us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.882447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.882460 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:14.882491 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.882494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.882534 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.882600 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:14.883009 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.883637 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.884564 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.884895 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.884989 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.885918 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:14.885920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:14.885922 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:14.885941 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:14.887172 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.887580 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.887582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.887583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.887655 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.887914 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.887917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.887918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.888068 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:14.888135 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.888386 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.888468 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1184}
I20260928 06:40:14.888651 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.881130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42257 (local address 127.8.151.195:59959)
0928 06:40:14.881361 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.881372 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.881436 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.882637 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.882640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.882828 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:14.882914 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.882919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.883755 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.883758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.884459 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.884468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.884984 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.884987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.885048 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.885446 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:14.885460 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:14.885799 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:14.887276 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.887278 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.887280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.887498 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.887730 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:14.887733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:14.887735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:14.887823 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:14.888146 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:14.888149 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:14.888240 (+ 91us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.888352 (+ 112us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.888482 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":83,"threads_started":1}
Sep 28 06:40:14 dist-test-slave-gtr8 krb5kdc[10484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 06:40:14 dist-test-slave-gtr8 krb5kdc[10484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:14.907016 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.900134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.8.151.254:42257)
0928 06:40:14.900236 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.900240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.900251 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:40:14.900283 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.900285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.900320 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.900385 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:14.900972 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.901456 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.902230 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.902433 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.902504 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.904504 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:14.904507 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:14.904509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:14.904530 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:14.905751 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.906095 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.906098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.906099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.906185 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:14.906373 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:14.906374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:14.906375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:14.906539 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:14.906630 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.906811 (+ 181us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.906901 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 06:40:14.909809 10529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:14.912843 10529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:14.926488 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.921085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42655 (local address 127.0.0.1:41990)
0928 06:40:14.921845 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.921895 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.922088 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.922713 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.922718 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.922739 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:14.922918 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.922924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.923932 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.923936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.925140 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.925148 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.925949 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.925953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.926071 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.926108 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.926209 (+ 101us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.926334 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":101,"spinlock_wait_cycles":6272,"thread_start_us":95,"threads_started":1}
I20260928 06:40:14.926491 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.921253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.8.151.194:42655)
0928 06:40:14.921562 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.921573 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.921980 (+ 407us) server_negotiation.cc:407] Connection header received
0928 06:40:14.922262 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.922268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.922476 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.922570 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:14.923022 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.923827 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.925276 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.925841 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.925936 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.926082 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.926246 (+ 164us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.926336 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":109,"threads_started":1}
I20260928 06:40:14.926841 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.920965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60226 (local address 127.8.151.193:45853)
0928 06:40:14.921402 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.921414 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.921973 (+ 559us) server_negotiation.cc:407] Connection header received
0928 06:40:14.922290 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.922296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.922487 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.922605 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:14.923076 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.923840 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.925318 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.926101 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.926193 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.926312 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.926584 (+ 272us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.926673 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":107,"threads_started":1}
I20260928 06:40:14.927206 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.920862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:45853 (local address 127.0.0.1:60226)
0928 06:40:14.921844 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.921894 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.922088 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.922775 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.922778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.922797 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:14.922933 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.922940 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.923947 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.923950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.925219 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.925229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.926368 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.926371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.926450 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.926472 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.926975 (+ 503us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.927049 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":65,"spinlock_wait_cycles":13312,"thread_start_us":68,"threads_started":1}
I20260928 06:40:14.928252 11034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.921684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48026 (local address 127.8.151.195:43599)
0928 06:40:14.922076 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.922087 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.922282 (+ 195us) server_negotiation.cc:407] Connection header received
0928 06:40:14.922486 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.922491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.922647 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.922788 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:14.923958 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.925126 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.926448 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.927240 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.927333 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.927461 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.928017 (+ 556us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.928106 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":100,"threads_started":1}
I20260928 06:40:14.928602 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.921544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:43599 (local address 127.0.0.1:48026)
0928 06:40:14.922208 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.922221 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.922352 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.922766 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.922768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.922781 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:14.922912 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.922918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.925231 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.925234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.926326 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.926337 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.927779 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.927782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.927873 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.927895 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.928439 (+ 544us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.928510 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":2,"spinlock_wait_cycles":11520,"thread_start_us":103,"threads_started":1}
I20260928 06:40:14.929718 10666 tablet_service.cc:1467] Processing CreateTablet for tablet dff8c48795254d64b73b76a48a0cfe60 (DEFAULT_TABLE table=test-table [id=cedccd2de8a94f148ca5a7a19c649800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:14.929585 10806 tablet_service.cc:1467] Processing CreateTablet for tablet dff8c48795254d64b73b76a48a0cfe60 (DEFAULT_TABLE table=test-table [id=cedccd2de8a94f148ca5a7a19c649800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:14.930697 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff8c48795254d64b73b76a48a0cfe60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:14.930749 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff8c48795254d64b73b76a48a0cfe60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:14.930641 10946 tablet_service.cc:1467] Processing CreateTablet for tablet dff8c48795254d64b73b76a48a0cfe60 (DEFAULT_TABLE table=test-table [id=cedccd2de8a94f148ca5a7a19c649800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:14.931509 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff8c48795254d64b73b76a48a0cfe60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:14.937800 11035 tablet_bootstrap.cc:492] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: Bootstrap starting.
I20260928 06:40:14.938827 11036 tablet_bootstrap.cc:492] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: Bootstrap starting.
I20260928 06:40:14.939459 11037 tablet_bootstrap.cc:492] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: Bootstrap starting.
I20260928 06:40:14.939915 11035 tablet_bootstrap.cc:654] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:14.940681 11035 log.cc:824] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:14.940953 11036 tablet_bootstrap.cc:654] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:14.941440 11037 tablet_bootstrap.cc:654] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:14.941715 11036 log.cc:824] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:14.942157 11037 log.cc:824] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:14.942600 11035 tablet_bootstrap.cc:492] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: No bootstrap required, opened a new log
I20260928 06:40:14.942785 11035 ts_tablet_manager.cc:1397] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260928 06:40:14.943871 11036 tablet_bootstrap.cc:492] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: No bootstrap required, opened a new log
I20260928 06:40:14.944051 11036 ts_tablet_manager.cc:1397] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260928 06:40:14.944079 11037 tablet_bootstrap.cc:492] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: No bootstrap required, opened a new log
I20260928 06:40:14.944221 11037 ts_tablet_manager.cc:1397] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260928 06:40:14.945721 11035 raft_consensus.cc:348] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } }
I20260928 06:40:14.945925 11035 raft_consensus.cc:374] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:14.945974 11035 raft_consensus.cc:729] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10db4b9e0d7047f899f6e89c55b91e26, State: Initialized, Role: FOLLOWER
I20260928 06:40:14.946376 11035 consensus_queue.cc:260] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } }
I20260928 06:40:14.947021 11012 heartbeater.cc:503] Master 127.8.151.254:42257 was elected leader, sending a full tablet report...
I20260928 06:40:14.947330 11035 ts_tablet_manager.cc:1428] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260928 06:40:14.947705 11037 raft_consensus.cc:348] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } }
I20260928 06:40:14.947705 11036 raft_consensus.cc:348] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } }
I20260928 06:40:14.947942 11037 raft_consensus.cc:374] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:14.947942 11036 raft_consensus.cc:374] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:14.947996 11036 raft_consensus.cc:729] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d21973ace6d4f4b9134cc426b07de88, State: Initialized, Role: FOLLOWER
I20260928 06:40:14.947996 11037 raft_consensus.cc:729] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cdbbc1620943d7b35e402f2427423f, State: Initialized, Role: FOLLOWER
I20260928 06:40:14.948400 11037 consensus_queue.cc:260] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } }
I20260928 06:40:14.948400 11036 consensus_queue.cc:260] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } }
I20260928 06:40:14.948997 10872 heartbeater.cc:503] Master 127.8.151.254:42257 was elected leader, sending a full tablet report...
I20260928 06:40:14.949261 10732 heartbeater.cc:503] Master 127.8.151.254:42257 was elected leader, sending a full tablet report...
I20260928 06:40:14.949299 11036 ts_tablet_manager.cc:1428] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260928 06:40:14.950139 11037 ts_tablet_manager.cc:1428] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260928 06:40:14.967432 10733 tablet.cc:2369] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:14.974830 11041 raft_consensus.cc:482] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:14.975030 11041 raft_consensus.cc:504] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } }
I20260928 06:40:14.976195 11041 leader_election.cc:288] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853), e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655)
I20260928 06:40:14.979892 10905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.976444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42655 (local address 127.8.151.195:35065)
0928 06:40:14.976622 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.976636 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.976709 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.976961 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.976963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.976977 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:14.977096 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.977101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.977841 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.977843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.978853 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.978863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.979572 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.979575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.979651 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.979671 (+ 20us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.979725 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.979781 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260928 06:40:14.979940 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.976540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:35065 (local address 127.8.151.194:42655)
0928 06:40:14.976657 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.976661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.976678 (+ 17us) server_negotiation.cc:407] Connection header received
0928 06:40:14.976810 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.976814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.976862 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.976930 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:14.977202 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.977742 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.978974 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.979461 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.979592 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.979642 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.979748 (+ 106us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.979813 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:40:14.980504 10826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff8c48795254d64b73b76a48a0cfe60" candidate_uuid: "10db4b9e0d7047f899f6e89c55b91e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cdbbc1620943d7b35e402f2427423f" is_pre_election: true
I20260928 06:40:14.980638 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.976492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:37419 (local address 127.8.151.193:45853)
0928 06:40:14.976608 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:14.976612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:14.976965 (+ 353us) server_negotiation.cc:407] Connection header received
0928 06:40:14.977054 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:14.977056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:14.977092 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:14.977153 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:14.977742 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.978240 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.979410 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.979958 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.980064 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.980109 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:14.980481 (+ 372us) server_negotiation.cc:299] Negotiation successful
0928 06:40:14.980551 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:40:14.980635 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:14.976447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:45853 (local address 127.8.151.195:37419)
0928 06:40:14.976876 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:14.976890 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:14.976974 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:14.977174 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:14.977177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:14.977205 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:14.977613 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.977619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.978347 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.978350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.979300 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:14.979314 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:14.980183 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:14.980186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:14.980344 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:14.980368 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 06:40:14.980441 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 06:40:14.980506 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":31,"thread_start_us":179,"threads_started":1}
I20260928 06:40:14.980830 10826 raft_consensus.cc:2454] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10db4b9e0d7047f899f6e89c55b91e26 in term 0.
I20260928 06:40:14.981211 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff8c48795254d64b73b76a48a0cfe60" candidate_uuid: "10db4b9e0d7047f899f6e89c55b91e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d21973ace6d4f4b9134cc426b07de88" is_pre_election: true
I20260928 06:40:14.981280 10897 leader_election.cc:302] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10db4b9e0d7047f899f6e89c55b91e26, e9cdbbc1620943d7b35e402f2427423f; no voters:
I20260928 06:40:14.981458 10686 raft_consensus.cc:2454] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10db4b9e0d7047f899f6e89c55b91e26 in term 0.
I20260928 06:40:14.981567 11041 raft_consensus.cc:2781] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:14.981648 11041 raft_consensus.cc:482] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:14.981696 11041 raft_consensus.cc:3037] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:14.982788 11041 raft_consensus.cc:504] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } }
I20260928 06:40:14.983093 11041 leader_election.cc:288] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [CANDIDATE]: Term 1 election: Requested vote from peers 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853), e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655)
I20260928 06:40:14.983420 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff8c48795254d64b73b76a48a0cfe60" candidate_uuid: "10db4b9e0d7047f899f6e89c55b91e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d21973ace6d4f4b9134cc426b07de88"
I20260928 06:40:14.983445 10826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff8c48795254d64b73b76a48a0cfe60" candidate_uuid: "10db4b9e0d7047f899f6e89c55b91e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cdbbc1620943d7b35e402f2427423f"
I20260928 06:40:14.983527 10686 raft_consensus.cc:3037] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:14.983541 10826 raft_consensus.cc:3037] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:14.984545 10686 raft_consensus.cc:2454] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10db4b9e0d7047f899f6e89c55b91e26 in term 1.
I20260928 06:40:14.984553 10826 raft_consensus.cc:2454] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10db4b9e0d7047f899f6e89c55b91e26 in term 1.
I20260928 06:40:14.984807 10897 leader_election.cc:302] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10db4b9e0d7047f899f6e89c55b91e26, e9cdbbc1620943d7b35e402f2427423f; no voters:
I20260928 06:40:14.984961 11041 raft_consensus.cc:2781] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:14.985168 11041 raft_consensus.cc:686] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 LEADER]: Becoming Leader. State: Replica: 10db4b9e0d7047f899f6e89c55b91e26, State: Running, Role: LEADER
I20260928 06:40:14.985450 11041 consensus_queue.cc:237] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } }
I20260928 06:40:14.988386 10527 catalog_manager.cc:5495] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195). New cstate: current_term: 1 leader_uuid: "10db4b9e0d7047f899f6e89c55b91e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } health_report { overall_health: UNKNOWN } } }
Sep 28 06:40:15 dist-test-slave-gtr8 krb5kdc[10484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577614, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:15.032541 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.025009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57318 (local address 127.8.151.254:42257)
0928 06:40:15.025159 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.025162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.025283 (+ 121us) server_negotiation.cc:407] Connection header received
0928 06:40:15.025381 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.025383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.025425 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.025483 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:15.026150 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.026650 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.027437 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.027638 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.027711 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.029651 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:15.029654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:15.029656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:15.029675 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:15.030943 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:15.031415 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:15.031417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:15.031419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:15.031495 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:15.031701 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:15.031703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:15.031704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:15.031872 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:15.031941 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.032273 (+ 332us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.032412 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":66}
I20260928 06:40:15.032543 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.024886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42257 (local address 127.0.0.1:57318)
0928 06:40:15.025180 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:15.025193 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:15.025291 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:15.025513 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:15.025515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:15.025958 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:15.026056 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.026061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.026731 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.026733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.027336 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.027345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.027725 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.027727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.027786 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.028260 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:15.028272 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:15.029536 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:15.031032 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:15.031040 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:15.031045 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:15.031330 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:15.031562 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:15.031564 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:15.031565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:15.031638 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:15.031985 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:15.032009 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:15.032118 (+ 109us) client_negotiation.cc:769] Sending connection context
0928 06:40:15.032227 (+ 109us) client_negotiation.cc:240] Negotiation successful
0928 06:40:15.032369 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":85,"threads_started":1}
I20260928 06:40:15.036384 10527 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:57318:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 06:40:15.037171 10527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:15.042078 10666 tablet_service.cc:1467] Processing CreateTablet for tablet bc8ad9444adb4afb9fd80c8f61f0ec02 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9745c0678c2a4a29bd8dd31109530473]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 06:40:15.042167 10946 tablet_service.cc:1467] Processing CreateTablet for tablet bc8ad9444adb4afb9fd80c8f61f0ec02 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9745c0678c2a4a29bd8dd31109530473]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 06:40:15.042315 10806 tablet_service.cc:1467] Processing CreateTablet for tablet bc8ad9444adb4afb9fd80c8f61f0ec02 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9745c0678c2a4a29bd8dd31109530473]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 06:40:15.042833 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8ad9444adb4afb9fd80c8f61f0ec02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:15.042833 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8ad9444adb4afb9fd80c8f61f0ec02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:15.042833 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8ad9444adb4afb9fd80c8f61f0ec02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:15.045475 11036 tablet_bootstrap.cc:492] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: Bootstrap starting.
I20260928 06:40:15.045568 11037 tablet_bootstrap.cc:492] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: Bootstrap starting.
I20260928 06:40:15.045568 11035 tablet_bootstrap.cc:492] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: Bootstrap starting.
I20260928 06:40:15.046274 11036 tablet_bootstrap.cc:654] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:15.046466 11037 tablet_bootstrap.cc:654] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:15.046466 11035 tablet_bootstrap.cc:654] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:15.047677 11037 tablet_bootstrap.cc:492] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: No bootstrap required, opened a new log
I20260928 06:40:15.047747 11037 ts_tablet_manager.cc:1397] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:15.047770 11036 tablet_bootstrap.cc:492] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: No bootstrap required, opened a new log
I20260928 06:40:15.047852 11036 ts_tablet_manager.cc:1397] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:15.048136 11037 raft_consensus.cc:348] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.048229 11037 raft_consensus.cc:374] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:15.048249 11037 raft_consensus.cc:729] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cdbbc1620943d7b35e402f2427423f, State: Initialized, Role: FOLLOWER
I20260928 06:40:15.048230 11036 raft_consensus.cc:348] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.048339 11036 raft_consensus.cc:374] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:15.048372 11036 raft_consensus.cc:729] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d21973ace6d4f4b9134cc426b07de88, State: Initialized, Role: FOLLOWER
I20260928 06:40:15.048352 11037 consensus_queue.cc:260] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.048492 11036 consensus_queue.cc:260] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.048573 11037 ts_tablet_manager.cc:1428] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 06:40:15.048594 11043 tablet_replica.cc:439] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } }
I20260928 06:40:15.048728 11043 tablet_replica.cc:442] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:15.048753 11036 ts_tablet_manager.cc:1428] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 06:40:15.048781 11042 tablet_replica.cc:439] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } }
I20260928 06:40:15.048923 11042 tablet_replica.cc:442] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:15.050098 11035 tablet_bootstrap.cc:492] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: No bootstrap required, opened a new log
I20260928 06:40:15.050192 11035 ts_tablet_manager.cc:1397] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260928 06:40:15.050582 11035 raft_consensus.cc:348] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.050676 11035 raft_consensus.cc:374] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:15.050705 11035 raft_consensus.cc:729] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10db4b9e0d7047f899f6e89c55b91e26, State: Initialized, Role: FOLLOWER
I20260928 06:40:15.050833 11035 consensus_queue.cc:260] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.051064 11035 ts_tablet_manager.cc:1428] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:15.051301 11041 tablet_replica.cc:439] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } }
I20260928 06:40:15.051446 11041 tablet_replica.cc:442] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: This TxnStatusTablet replica's current role is: FOLLOWER
W20260928 06:40:15.126031 11013 tablet.cc:2369] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 06:40:15.172214 10873 tablet.cc:2369] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:15.196379 11042 raft_consensus.cc:482] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:15.196538 11042 raft_consensus.cc:504] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.197638 11042 leader_election.cc:288] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655), 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599)
I20260928 06:40:15.201999 11034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.198050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:42327 (local address 127.8.151.195:43599)
0928 06:40:15.198162 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.198165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.198290 (+ 125us) server_negotiation.cc:407] Connection header received
0928 06:40:15.198444 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.198447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.198491 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.198557 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:15.198828 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.199502 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.200936 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.201435 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.201548 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.201622 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.201794 (+ 172us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.201882 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:40:15.202011 11056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.197905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:43599 (local address 127.8.151.193:42327)
0928 06:40:15.198213 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:15.198224 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:15.198294 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:15.198568 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:15.198570 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:15.198621 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:15.198718 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.198725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.199621 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.199623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.200828 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.200839 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.201569 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.201572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.201668 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.201696 (+ 28us) client_negotiation.cc:769] Sending connection context
0928 06:40:15.201766 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 06:40:15.201834 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"spinlock_wait_cycles":37504,"thread_start_us":76,"threads_started":1}
I20260928 06:40:15.202374 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.197940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:59473 (local address 127.8.151.194:42655)
0928 06:40:15.198061 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.198066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.198252 (+ 186us) server_negotiation.cc:407] Connection header received
0928 06:40:15.198382 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.198385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.198434 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.198513 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:15.199252 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.199873 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.201178 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.201743 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.201860 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.201904 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.202192 (+ 288us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.202270 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 06:40:15.202425 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.197862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42655 (local address 127.8.151.193:59473)
0928 06:40:15.198163 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:15.198176 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:15.198288 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:15.198534 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:15.198537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:15.198559 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:15.199119 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.199127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.199989 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.199992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.201083 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.201090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.201864 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.201868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.202056 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.202078 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 06:40:15.202151 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 06:40:15.202217 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"spinlock_wait_cycles":3456,"thread_start_us":81,"threads_started":1}
I20260928 06:40:15.202678 10966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8ad9444adb4afb9fd80c8f61f0ec02" candidate_uuid: "5d21973ace6d4f4b9134cc426b07de88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10db4b9e0d7047f899f6e89c55b91e26" is_pre_election: true
I20260928 06:40:15.202767 10826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8ad9444adb4afb9fd80c8f61f0ec02" candidate_uuid: "5d21973ace6d4f4b9134cc426b07de88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cdbbc1620943d7b35e402f2427423f" is_pre_election: true
I20260928 06:40:15.202890 10826 raft_consensus.cc:2454] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d21973ace6d4f4b9134cc426b07de88 in term 0.
I20260928 06:40:15.202924 10966 raft_consensus.cc:2454] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d21973ace6d4f4b9134cc426b07de88 in term 0.
I20260928 06:40:15.203280 10617 leader_election.cc:302] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d21973ace6d4f4b9134cc426b07de88, e9cdbbc1620943d7b35e402f2427423f; no voters:
I20260928 06:40:15.203598 11042 raft_consensus.cc:2781] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:15.203671 11042 raft_consensus.cc:482] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:15.203711 11042 raft_consensus.cc:3037] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:15.204785 11042 raft_consensus.cc:504] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.205109 11042 leader_election.cc:288] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [CANDIDATE]: Term 1 election: Requested vote from peers e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655), 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599)
I20260928 06:40:15.205415 10826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8ad9444adb4afb9fd80c8f61f0ec02" candidate_uuid: "5d21973ace6d4f4b9134cc426b07de88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cdbbc1620943d7b35e402f2427423f"
I20260928 06:40:15.205542 10826 raft_consensus.cc:3037] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:15.205554 10966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8ad9444adb4afb9fd80c8f61f0ec02" candidate_uuid: "5d21973ace6d4f4b9134cc426b07de88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10db4b9e0d7047f899f6e89c55b91e26"
I20260928 06:40:15.205670 10966 raft_consensus.cc:3037] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:15.206496 10826 raft_consensus.cc:2454] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d21973ace6d4f4b9134cc426b07de88 in term 1.
I20260928 06:40:15.206594 10966 raft_consensus.cc:2454] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d21973ace6d4f4b9134cc426b07de88 in term 1.
I20260928 06:40:15.206748 10617 leader_election.cc:302] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d21973ace6d4f4b9134cc426b07de88, e9cdbbc1620943d7b35e402f2427423f; no voters:
I20260928 06:40:15.206944 11042 raft_consensus.cc:2781] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:15.207163 11042 raft_consensus.cc:686] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [term 1 LEADER]: Becoming Leader. State: Replica: 5d21973ace6d4f4b9134cc426b07de88, State: Running, Role: LEADER
I20260928 06:40:15.207494 11042 consensus_queue.cc:237] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.209728 11058 tablet_replica.cc:439] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: TxnStatusTablet state changed. Reason: New leader 5d21973ace6d4f4b9134cc426b07de88. Latest consensus state: current_term: 1 leader_uuid: "5d21973ace6d4f4b9134cc426b07de88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } }
I20260928 06:40:15.209836 11058 tablet_replica.cc:442] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: This TxnStatusTablet replica's current role is: LEADER
I20260928 06:40:15.210953 10527 catalog_manager.cc:5495] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193). New cstate: current_term: 1 leader_uuid: "5d21973ace6d4f4b9134cc426b07de88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } health_report { overall_health: UNKNOWN } } }
I20260928 06:40:15.215309 11060 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 06:40:15.215499 11060 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 06:40:15.239904 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.236201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:45853 (local address 127.0.0.1:60238)
0928 06:40:15.236431 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:15.236448 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:15.236566 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:15.236865 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:15.236868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:15.236875 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:15.236988 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.236994 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.237690 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.237693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.238305 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.238313 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.238723 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.238727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.238828 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.238878 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.239601 (+ 723us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.239618 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 06:40:15.239680 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 06:40:15.239762 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260928 06:40:15.239884 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.236311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.8.151.193:45853)
0928 06:40:15.236460 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.236464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.236496 (+ 32us) server_negotiation.cc:407] Connection header received
0928 06:40:15.236697 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.236699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.236752 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.236830 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:15.237084 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.237588 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.238415 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.238623 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.238707 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.239009 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.239513 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.239617 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.239691 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.239741 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 06:40:15.245455 10966 raft_consensus.cc:1261] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Refusing update from remote peer 5d21973ace6d4f4b9134cc426b07de88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:15.246078 11058 consensus_queue.cc:1035] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:15.246234 10826 raft_consensus.cc:1261] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Refusing update from remote peer 5d21973ace6d4f4b9134cc426b07de88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:15.246752 11058 consensus_queue.cc:1035] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:15.249333 11041 tablet_replica.cc:439] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: TxnStatusTablet state changed. Reason: New leader 5d21973ace6d4f4b9134cc426b07de88. Latest consensus state: current_term: 1 leader_uuid: "5d21973ace6d4f4b9134cc426b07de88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } }
I20260928 06:40:15.249452 11041 tablet_replica.cc:442] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:15.250798 11043 tablet_replica.cc:439] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: TxnStatusTablet state changed. Reason: New leader 5d21973ace6d4f4b9134cc426b07de88. Latest consensus state: current_term: 1 leader_uuid: "5d21973ace6d4f4b9134cc426b07de88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } }
I20260928 06:40:15.250910 11043 tablet_replica.cc:442] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:15.253453 11058 tablet_replica.cc:439] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5d21973ace6d4f4b9134cc426b07de88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } }
I20260928 06:40:15.253574 11058 tablet_replica.cc:442] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: This TxnStatusTablet replica's current role is: LEADER
I20260928 06:40:15.253558 11041 tablet_replica.cc:439] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5d21973ace6d4f4b9134cc426b07de88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } }
I20260928 06:40:15.253644 11041 tablet_replica.cc:442] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 10db4b9e0d7047f899f6e89c55b91e26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:15.253692 11058 tablet_replica.cc:439] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5d21973ace6d4f4b9134cc426b07de88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } }
I20260928 06:40:15.253767 11058 tablet_replica.cc:442] T bc8ad9444adb4afb9fd80c8f61f0ec02 P 5d21973ace6d4f4b9134cc426b07de88: This TxnStatusTablet replica's current role is: LEADER
I20260928 06:40:15.255267 11063 mvcc.cc:205] Tried to move back new op lower bound from 7334205912038748160 to 7334205911892623360. Current Snapshot: MvccSnapshot[applied={T|T < 7334205912038748160}]
I20260928 06:40:15.255281 11043 tablet_replica.cc:439] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5d21973ace6d4f4b9134cc426b07de88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } } }
I20260928 06:40:15.255384 11043 tablet_replica.cc:442] T bc8ad9444adb4afb9fd80c8f61f0ec02 P e9cdbbc1620943d7b35e402f2427423f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:15.257017 11064 mvcc.cc:205] Tried to move back new op lower bound from 7334205912038748160 to 7334205911892623360. Current Snapshot: MvccSnapshot[applied={T|T < 7334205912038748160}]
I20260928 06:40:15.265069 11034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.261208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48038 (local address 127.8.151.195:43599)
0928 06:40:15.261326 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.261330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.261345 (+ 15us) server_negotiation.cc:407] Connection header received
0928 06:40:15.261441 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.261444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.261501 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.261591 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:15.261856 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.262646 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.263549 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.263888 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.263979 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.264226 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.264711 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.264816 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.264879 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.264933 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 06:40:15.276373 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.272644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:45853 (local address 127.8.151.195:57249)
0928 06:40:15.272775 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:15.272795 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:15.272881 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:15.273120 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:15.273123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:15.273130 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:15.273244 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.273249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.274263 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.274267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.274991 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.275000 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.275583 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.275586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.275665 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.275750 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.276087 (+ 337us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.276094 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 06:40:15.276170 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 06:40:15.276250 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 06:40:15.276379 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.272771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:57249 (local address 127.8.151.193:45853)
0928 06:40:15.272896 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.272900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.272914 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:40:15.272961 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.272964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.273011 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.273081 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:15.273343 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.274104 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.275102 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.275472 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.275572 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.275897 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.275981 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.276047 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.276184 (+ 137us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.276235 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 06:40:15.285034 10826 raft_consensus.cc:1261] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Refusing update from remote peer 10db4b9e0d7047f899f6e89c55b91e26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:15.285364 10686 raft_consensus.cc:1261] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Refusing update from remote peer 10db4b9e0d7047f899f6e89c55b91e26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:15.285478 11045 consensus_queue.cc:1035] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:15.285873 11041 consensus_queue.cc:1035] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:15.289508 11066 mvcc.cc:205] Tried to move back new op lower bound from 7334205912198635520 to 7334205910982803456. Current Snapshot: MvccSnapshot[applied={T|T < 7334205912198635520}]
I20260928 06:40:15.306824 11034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.303089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:52147 (local address 127.8.151.195:43599)
0928 06:40:15.303253 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.303259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.303417 (+ 158us) server_negotiation.cc:407] Connection header received
0928 06:40:15.303556 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.303560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.303619 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.303696 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:15.303975 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.304487 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.305785 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.305998 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.306081 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.306383 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.306454 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.306511 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.306667 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.306718 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260928 06:40:15.306838 11080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.302974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:43599 (local address 127.8.151.193:52147)
0928 06:40:15.303331 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:15.303351 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:15.303461 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:15.303712 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:15.303716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:15.303724 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:15.303855 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.303862 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.304596 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.304599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.305616 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.305627 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.306096 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.306100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.306197 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.306248 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.306538 (+ 290us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.306544 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:40:15.306610 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 06:40:15.306686 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":106,"threads_started":1}
I20260928 06:40:15.329593 10529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:15.329799 10529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:15.332520 10529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07: Sending DeleteTablet for 3 replicas of tablet dff8c48795254d64b73b76a48a0cfe60
I20260928 06:40:15.333352 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet dff8c48795254d64b73b76a48a0cfe60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:15 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:60226
I20260928 06:40:15.333432 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet dff8c48795254d64b73b76a48a0cfe60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:15 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:41990
I20260928 06:40:15.334009 11084 tablet_replica.cc:330] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: stopping tablet replica
I20260928 06:40:15.334012 10946 tablet_service.cc:1514] Processing DeleteTablet for tablet dff8c48795254d64b73b76a48a0cfe60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:15 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48026
I20260928 06:40:15.334443 11084 raft_consensus.cc:2229] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:15.334712 11084 raft_consensus.cc:2258] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:15.335127 10529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:15.335566 10529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:15.336382 11084 ts_tablet_manager.cc:1905] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:15.338828 11085 tablet_replica.cc:330] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: stopping tablet replica
I20260928 06:40:15.339161 11085 raft_consensus.cc:2229] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:15.339519 11085 raft_consensus.cc:2258] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:15.340355 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb3fcdcb62049898047f74005c76acf (DEFAULT_TABLE table=test-table [id=f81ef66d85d946579077733156567d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:15.339967 11084 ts_tablet_manager.cc:1918] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:15.340703 11084 log.cc:1196] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/dff8c48795254d64b73b76a48a0cfe60
I20260928 06:40:15.340770 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb3fcdcb62049898047f74005c76acf (DEFAULT_TABLE table=test-table [id=f81ef66d85d946579077733156567d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:15.341017 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb3fcdcb62049898047f74005c76acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:15.341221 11084 ts_tablet_manager.cc:1939] T dff8c48795254d64b73b76a48a0cfe60 P e9cdbbc1620943d7b35e402f2427423f: Deleting consensus metadata
I20260928 06:40:15.343062 11037 tablet_bootstrap.cc:492] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f: Bootstrap starting.
I20260928 06:40:15.343227 10513 catalog_manager.cc:4841] TS e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655): tablet dff8c48795254d64b73b76a48a0cfe60 (table test-table [id=cedccd2de8a94f148ca5a7a19c649800]) successfully deleted
I20260928 06:40:15.344094 11037 tablet_bootstrap.cc:654] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:15.344147 10946 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb3fcdcb62049898047f74005c76acf (DEFAULT_TABLE table=test-table [id=f81ef66d85d946579077733156567d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:15.344393 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb3fcdcb62049898047f74005c76acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:15.345487 11037 tablet_bootstrap.cc:492] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f: No bootstrap required, opened a new log
I20260928 06:40:15.345564 11037 ts_tablet_manager.cc:1397] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 06:40:15.345983 11037 raft_consensus.cc:348] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.346094 11037 raft_consensus.cc:374] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:15.346125 11037 raft_consensus.cc:729] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cdbbc1620943d7b35e402f2427423f, State: Initialized, Role: FOLLOWER
I20260928 06:40:15.346236 11037 consensus_queue.cc:260] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.346483 11037 ts_tablet_manager.cc:1428] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:15.346585 11035 tablet_bootstrap.cc:492] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26: Bootstrap starting.
I20260928 06:40:15.347464 11035 tablet_bootstrap.cc:654] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:15.348906 11035 tablet_bootstrap.cc:492] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26: No bootstrap required, opened a new log
I20260928 06:40:15.348980 11035 ts_tablet_manager.cc:1397] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 06:40:15.349422 11035 raft_consensus.cc:348] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.349530 11035 raft_consensus.cc:374] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:15.349558 11035 raft_consensus.cc:729] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10db4b9e0d7047f899f6e89c55b91e26, State: Initialized, Role: FOLLOWER
I20260928 06:40:15.349675 11035 consensus_queue.cc:260] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.349924 11035 ts_tablet_manager.cc:1428] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260928 06:40:15.352309 11085 ts_tablet_manager.cc:1905] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:15.353940 11083 tablet_replica.cc:330] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: stopping tablet replica
I20260928 06:40:15.354305 11083 raft_consensus.cc:2229] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:15.354403 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb3fcdcb62049898047f74005c76acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:15.354480 11083 raft_consensus.cc:2258] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:15.355854 11083 ts_tablet_manager.cc:1905] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:15.357046 11036 tablet_bootstrap.cc:492] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88: Bootstrap starting.
I20260928 06:40:15.357955 11036 tablet_bootstrap.cc:654] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:15.358521 11085 ts_tablet_manager.cc:1918] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:15.358616 11085 log.cc:1196] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/dff8c48795254d64b73b76a48a0cfe60
I20260928 06:40:15.358901 11085 ts_tablet_manager.cc:1939] T dff8c48795254d64b73b76a48a0cfe60 P 10db4b9e0d7047f899f6e89c55b91e26: Deleting consensus metadata
I20260928 06:40:15.360011 10516 catalog_manager.cc:4841] TS 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599): tablet dff8c48795254d64b73b76a48a0cfe60 (table test-table [id=cedccd2de8a94f148ca5a7a19c649800]) successfully deleted
I20260928 06:40:15.360661 11036 tablet_bootstrap.cc:492] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88: No bootstrap required, opened a new log
I20260928 06:40:15.360743 11036 ts_tablet_manager.cc:1397] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260928 06:40:15.361169 11036 raft_consensus.cc:348] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.361305 11036 raft_consensus.cc:374] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:15.361340 11036 raft_consensus.cc:729] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d21973ace6d4f4b9134cc426b07de88, State: Initialized, Role: FOLLOWER
I20260928 06:40:15.361457 11036 consensus_queue.cc:260] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.361485 11083 ts_tablet_manager.cc:1918] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:15.361660 11083 log.cc:1196] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/dff8c48795254d64b73b76a48a0cfe60
I20260928 06:40:15.362012 11083 ts_tablet_manager.cc:1939] T dff8c48795254d64b73b76a48a0cfe60 P 5d21973ace6d4f4b9134cc426b07de88: Deleting consensus metadata
I20260928 06:40:15.363396 10516 catalog_manager.cc:4841] TS 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853): tablet dff8c48795254d64b73b76a48a0cfe60 (table test-table [id=cedccd2de8a94f148ca5a7a19c649800]) successfully deleted
I20260928 06:40:15.361697 11036 ts_tablet_manager.cc:1428] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:15.405000 11043 raft_consensus.cc:482] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:15.405151 11043 raft_consensus.cc:504] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.406366 11043 leader_election.cc:288] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853), 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599)
I20260928 06:40:15.410635 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.406691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:44275 (local address 127.8.151.193:45853)
0928 06:40:15.406818 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.406822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.406919 (+ 97us) server_negotiation.cc:407] Connection header received
0928 06:40:15.407122 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.407125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.407199 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.407247 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:15.407617 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.408178 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.409501 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.410111 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.410226 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.410308 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.410460 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.410537 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 06:40:15.410647 11089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.406562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:45853 (local address 127.8.151.194:44275)
0928 06:40:15.406811 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:15.406847 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:15.406997 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:15.407392 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:15.407395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:15.407408 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:15.407520 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.407525 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.408277 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.408280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.409376 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.409386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.410227 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.410231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.410333 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.410356 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:40:15.410423 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 06:40:15.410496 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
I20260928 06:40:15.410851 11034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.407024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:57579 (local address 127.8.151.195:43599)
0928 06:40:15.407131 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.407156 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.407378 (+ 222us) server_negotiation.cc:407] Connection header received
0928 06:40:15.407471 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.407474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.407523 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.407597 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:15.407834 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.408583 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.409793 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.410176 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.410281 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.410332 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.410660 (+ 328us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.410734 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 06:40:15.411118 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb3fcdcb62049898047f74005c76acf" candidate_uuid: "e9cdbbc1620943d7b35e402f2427423f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d21973ace6d4f4b9134cc426b07de88" is_pre_election: true
I20260928 06:40:15.411247 11090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.406923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:43599 (local address 127.8.151.194:57579)
0928 06:40:15.407291 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:15.407303 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:15.407394 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:15.407626 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:15.407629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:15.407645 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:15.407740 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.407745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.408694 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.408697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.409682 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.409689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.410405 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.410409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.410521 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.410544 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:40:15.411022 (+ 478us) client_negotiation.cc:240] Negotiation successful
0928 06:40:15.411096 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":99,"threads_started":1}
I20260928 06:40:15.411273 10686 raft_consensus.cc:2454] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cdbbc1620943d7b35e402f2427423f in term 0.
I20260928 06:40:15.411574 10760 leader_election.cc:302] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d21973ace6d4f4b9134cc426b07de88, e9cdbbc1620943d7b35e402f2427423f; no voters:
I20260928 06:40:15.411666 10966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb3fcdcb62049898047f74005c76acf" candidate_uuid: "e9cdbbc1620943d7b35e402f2427423f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10db4b9e0d7047f899f6e89c55b91e26" is_pre_election: true
I20260928 06:40:15.411780 10966 raft_consensus.cc:2454] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cdbbc1620943d7b35e402f2427423f in term 0.
I20260928 06:40:15.412060 11043 raft_consensus.cc:2781] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:15.412118 11043 raft_consensus.cc:482] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:15.412154 11043 raft_consensus.cc:3037] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:15.413334 11043 raft_consensus.cc:504] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.413684 11043 leader_election.cc:288] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [CANDIDATE]: Term 1 election: Requested vote from peers 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853), 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599)
I20260928 06:40:15.414110 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb3fcdcb62049898047f74005c76acf" candidate_uuid: "e9cdbbc1620943d7b35e402f2427423f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d21973ace6d4f4b9134cc426b07de88"
I20260928 06:40:15.414139 10966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb3fcdcb62049898047f74005c76acf" candidate_uuid: "e9cdbbc1620943d7b35e402f2427423f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10db4b9e0d7047f899f6e89c55b91e26"
I20260928 06:40:15.414248 10966 raft_consensus.cc:3037] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:15.414248 10686 raft_consensus.cc:3037] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:15.415417 10686 raft_consensus.cc:2454] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cdbbc1620943d7b35e402f2427423f in term 1.
I20260928 06:40:15.415474 10966 raft_consensus.cc:2454] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cdbbc1620943d7b35e402f2427423f in term 1.
I20260928 06:40:15.415728 10760 leader_election.cc:302] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d21973ace6d4f4b9134cc426b07de88, e9cdbbc1620943d7b35e402f2427423f; no voters:
I20260928 06:40:15.415982 11043 raft_consensus.cc:2781] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:15.416157 11043 raft_consensus.cc:686] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 1 LEADER]: Becoming Leader. State: Replica: e9cdbbc1620943d7b35e402f2427423f, State: Running, Role: LEADER
I20260928 06:40:15.416397 11043 consensus_queue.cc:237] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:15.418642 10529 catalog_manager.cc:5495] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cdbbc1620943d7b35e402f2427423f (127.8.151.194). New cstate: current_term: 1 leader_uuid: "e9cdbbc1620943d7b35e402f2427423f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } health_report { overall_health: UNKNOWN } } }
I20260928 06:40:15.440996 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.437098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42004 (local address 127.8.151.194:42655)
0928 06:40:15.437194 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.437198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.437209 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:40:15.437365 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.437367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.437413 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.437488 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:15.437709 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.438182 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.439463 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.439776 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.439877 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.440124 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.440659 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.440777 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.440809 (+ 32us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.440863 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 06:40:15.451351 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.447938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:38219 (local address 127.8.151.193:45853)
0928 06:40:15.448073 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.448078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.448174 (+ 96us) server_negotiation.cc:407] Connection header received
0928 06:40:15.448305 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.448309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.448396 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.448486 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:15.448679 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.449461 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.450259 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.450585 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.450688 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.450905 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.450988 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.451068 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.451129 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.451204 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 06:40:15.451745 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.447829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:45853 (local address 127.8.151.194:38219)
0928 06:40:15.448096 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:15.448108 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:15.448189 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:15.448486 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:15.448488 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:15.448494 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:15.448588 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.448591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.449551 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.449553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.450137 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.450144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.450684 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.450686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.450747 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.450785 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.451065 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.451068 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 06:40:15.451102 (+ 34us) client_negotiation.cc:240] Negotiation successful
0928 06:40:15.451189 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
I20260928 06:40:15.458062 10686 raft_consensus.cc:1261] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Refusing update from remote peer e9cdbbc1620943d7b35e402f2427423f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:15.458091 10966 raft_consensus.cc:1261] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Refusing update from remote peer e9cdbbc1620943d7b35e402f2427423f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:15.458492 11043 consensus_queue.cc:1035] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:15.458714 11091 consensus_queue.cc:1035] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [LEADER]: Connected to new peer: Peer: permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:15.477648 11080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.472847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42655 (local address 127.8.151.193:34675)
0928 06:40:15.473001 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:15.473015 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:15.473110 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:15.474547 (+ 1437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:15.474550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:15.474559 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:15.474681 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.474688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.475826 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.475846 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.476547 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:15.476557 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.477013 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.477016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:15.477097 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.477123 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.477404 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.477411 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 06:40:15.477463 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 06:40:15.477547 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260928 06:40:15.477962 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:15.473311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:34675 (local address 127.8.151.194:42655)
0928 06:40:15.474329 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:15.474334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:15.474347 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:15.474403 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:15.474406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:15.474451 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:15.474529 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:15.474885 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.475727 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.476655 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:15.476917 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:15.477002 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:15.477220 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:15.477308 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:15.477367 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:15.477804 (+ 437us) server_negotiation.cc:299] Negotiation successful
0928 06:40:15.477855 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945}
W20260928 06:40:15.478581 11066 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 06:40:17.041939 10529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:17.042093 10529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:17.044467 10529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07: Sending DeleteTablet for 3 replicas of tablet 5fb3fcdcb62049898047f74005c76acf
I20260928 06:40:17.044965 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb3fcdcb62049898047f74005c76acf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:17 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:41990
I20260928 06:40:17.044965 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb3fcdcb62049898047f74005c76acf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:17 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:60226
I20260928 06:40:17.045073 10946 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb3fcdcb62049898047f74005c76acf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:17 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48026
I20260928 06:40:17.045305 11123 tablet_replica.cc:330] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f: stopping tablet replica
I20260928 06:40:17.045426 11123 raft_consensus.cc:2229] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:17.046018 11122 tablet_replica.cc:330] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88: stopping tablet replica
I20260928 06:40:17.046129 11122 raft_consensus.cc:2229] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:17.046234 11122 raft_consensus.cc:2258] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:17.046504 11123 raft_consensus.cc:2258] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:17.046835 11122 ts_tablet_manager.cc:1905] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:17.047034 11123 ts_tablet_manager.cc:1905] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:17.048346 11124 tablet_replica.cc:330] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26: stopping tablet replica
I20260928 06:40:17.048478 11124 raft_consensus.cc:2229] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:17.048585 11124 raft_consensus.cc:2258] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:17.049198 11124 ts_tablet_manager.cc:1905] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:17.049405 11122 ts_tablet_manager.cc:1918] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 06:40:17.049407 11123 ts_tablet_manager.cc:1918] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 06:40:17.049481 11122 log.cc:1196] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/5fb3fcdcb62049898047f74005c76acf
I20260928 06:40:17.049481 11123 log.cc:1196] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/5fb3fcdcb62049898047f74005c76acf
I20260928 06:40:17.049721 11122 ts_tablet_manager.cc:1939] T 5fb3fcdcb62049898047f74005c76acf P 5d21973ace6d4f4b9134cc426b07de88: Deleting consensus metadata
I20260928 06:40:17.049726 11123 ts_tablet_manager.cc:1939] T 5fb3fcdcb62049898047f74005c76acf P e9cdbbc1620943d7b35e402f2427423f: Deleting consensus metadata
I20260928 06:40:17.050488 10513 catalog_manager.cc:4841] TS e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655): tablet 5fb3fcdcb62049898047f74005c76acf (table test-table [id=f81ef66d85d946579077733156567d72]) successfully deleted
I20260928 06:40:17.050501 10516 catalog_manager.cc:4841] TS 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853): tablet 5fb3fcdcb62049898047f74005c76acf (table test-table [id=f81ef66d85d946579077733156567d72]) successfully deleted
I20260928 06:40:17.051753 11124 ts_tablet_manager.cc:1918] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 06:40:17.051813 11124 log.cc:1196] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/5fb3fcdcb62049898047f74005c76acf
I20260928 06:40:17.052016 11124 ts_tablet_manager.cc:1939] T 5fb3fcdcb62049898047f74005c76acf P 10db4b9e0d7047f899f6e89c55b91e26: Deleting consensus metadata
I20260928 06:40:17.052605 10516 catalog_manager.cc:4841] TS 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599): tablet 5fb3fcdcb62049898047f74005c76acf (table test-table [id=f81ef66d85d946579077733156567d72]) successfully deleted
Sep 28 06:40:17 dist-test-slave-gtr8 krb5kdc[10484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 06:40:17 dist-test-slave-gtr8 krb5kdc[10484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:17.070749 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:17.062739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.8.151.254:42257)
0928 06:40:17.063033 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:17.063039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:17.063053 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:40:17.063120 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:17.063122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:17.063192 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:17.063268 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:17.063920 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:17.064457 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:17.065262 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:17.065482 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:17.065547 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:17.067755 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:17.067758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:17.067760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:17.067781 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:17.069062 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:17.069443 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:17.069446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:17.069447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:17.069532 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:17.069873 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:17.069877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:17.069879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:17.070079 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:17.070205 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:17.070457 (+ 252us) server_negotiation.cc:299] Negotiation successful
0928 06:40:17.070583 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":121,"threads_started":1}
I20260928 06:40:17.073354 10529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:17.073804 10529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:17.078367 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 549b0b6f774f40c38ec5d57837f69c4f (DEFAULT_TABLE table=test-table [id=c7b5508c4d8a4192baf1f73da57607d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:17.078425 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 549b0b6f774f40c38ec5d57837f69c4f (DEFAULT_TABLE table=test-table [id=c7b5508c4d8a4192baf1f73da57607d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:17.078653 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 549b0b6f774f40c38ec5d57837f69c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:17.078672 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 549b0b6f774f40c38ec5d57837f69c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:17.078682 10946 tablet_service.cc:1467] Processing CreateTablet for tablet 549b0b6f774f40c38ec5d57837f69c4f (DEFAULT_TABLE table=test-table [id=c7b5508c4d8a4192baf1f73da57607d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:17.078929 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 549b0b6f774f40c38ec5d57837f69c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:17.080835 11133 tablet_bootstrap.cc:492] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f: Bootstrap starting.
I20260928 06:40:17.081313 11134 tablet_bootstrap.cc:492] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26: Bootstrap starting.
I20260928 06:40:17.081759 11133 tablet_bootstrap.cc:654] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:17.082080 11134 tablet_bootstrap.cc:654] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:17.082666 11135 tablet_bootstrap.cc:492] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88: Bootstrap starting.
I20260928 06:40:17.082989 11133 tablet_bootstrap.cc:492] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f: No bootstrap required, opened a new log
I20260928 06:40:17.083092 11133 ts_tablet_manager.cc:1397] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:17.083127 11134 tablet_bootstrap.cc:492] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26: No bootstrap required, opened a new log
I20260928 06:40:17.083310 11134 ts_tablet_manager.cc:1397] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:17.083451 11135 tablet_bootstrap.cc:654] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:17.083573 11133 raft_consensus.cc:348] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.083683 11133 raft_consensus.cc:374] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:17.083716 11133 raft_consensus.cc:729] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cdbbc1620943d7b35e402f2427423f, State: Initialized, Role: FOLLOWER
I20260928 06:40:17.083717 11134 raft_consensus.cc:348] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.083804 11134 raft_consensus.cc:374] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:17.083832 11134 raft_consensus.cc:729] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10db4b9e0d7047f899f6e89c55b91e26, State: Initialized, Role: FOLLOWER
I20260928 06:40:17.083879 11133 consensus_queue.cc:260] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.083945 11134 consensus_queue.cc:260] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.084136 11133 ts_tablet_manager.cc:1428] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:17.084295 11134 ts_tablet_manager.cc:1428] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260928 06:40:17.085055 11135 tablet_bootstrap.cc:492] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88: No bootstrap required, opened a new log
I20260928 06:40:17.085145 11135 ts_tablet_manager.cc:1397] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 06:40:17.085574 11135 raft_consensus.cc:348] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.085672 11135 raft_consensus.cc:374] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:17.085702 11135 raft_consensus.cc:729] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d21973ace6d4f4b9134cc426b07de88, State: Initialized, Role: FOLLOWER
I20260928 06:40:17.085840 11135 consensus_queue.cc:260] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.086179 11135 ts_tablet_manager.cc:1428] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:17.170320 11042 raft_consensus.cc:482] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:17.170465 11042 raft_consensus.cc:504] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.170938 11042 leader_election.cc:288] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655), 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599)
I20260928 06:40:17.171454 10966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "549b0b6f774f40c38ec5d57837f69c4f" candidate_uuid: "5d21973ace6d4f4b9134cc426b07de88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10db4b9e0d7047f899f6e89c55b91e26" is_pre_election: true
I20260928 06:40:17.171453 10826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "549b0b6f774f40c38ec5d57837f69c4f" candidate_uuid: "5d21973ace6d4f4b9134cc426b07de88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cdbbc1620943d7b35e402f2427423f" is_pre_election: true
I20260928 06:40:17.171614 10826 raft_consensus.cc:2454] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d21973ace6d4f4b9134cc426b07de88 in term 0.
I20260928 06:40:17.171628 10966 raft_consensus.cc:2454] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d21973ace6d4f4b9134cc426b07de88 in term 0.
I20260928 06:40:17.171964 10617 leader_election.cc:302] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 10db4b9e0d7047f899f6e89c55b91e26, 5d21973ace6d4f4b9134cc426b07de88, e9cdbbc1620943d7b35e402f2427423f; no voters:
I20260928 06:40:17.172163 11042 raft_consensus.cc:2781] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:17.172227 11042 raft_consensus.cc:482] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:17.172261 11042 raft_consensus.cc:3037] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:17.173341 11042 raft_consensus.cc:504] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.173671 11042 leader_election.cc:288] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [CANDIDATE]: Term 1 election: Requested vote from peers e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655), 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599)
I20260928 06:40:17.174041 10966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "549b0b6f774f40c38ec5d57837f69c4f" candidate_uuid: "5d21973ace6d4f4b9134cc426b07de88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10db4b9e0d7047f899f6e89c55b91e26"
I20260928 06:40:17.174171 10966 raft_consensus.cc:3037] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:17.174194 10826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "549b0b6f774f40c38ec5d57837f69c4f" candidate_uuid: "5d21973ace6d4f4b9134cc426b07de88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cdbbc1620943d7b35e402f2427423f"
I20260928 06:40:17.174324 10826 raft_consensus.cc:3037] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:17.175422 10966 raft_consensus.cc:2454] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d21973ace6d4f4b9134cc426b07de88 in term 1.
I20260928 06:40:17.175422 10826 raft_consensus.cc:2454] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d21973ace6d4f4b9134cc426b07de88 in term 1.
I20260928 06:40:17.175699 10617 leader_election.cc:302] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d21973ace6d4f4b9134cc426b07de88, e9cdbbc1620943d7b35e402f2427423f; no voters:
I20260928 06:40:17.175868 11042 raft_consensus.cc:2781] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:17.175957 11042 raft_consensus.cc:686] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 1 LEADER]: Becoming Leader. State: Replica: 5d21973ace6d4f4b9134cc426b07de88, State: Running, Role: LEADER
I20260928 06:40:17.176092 11042 consensus_queue.cc:237] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.177889 10528 catalog_manager.cc:5495] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193). New cstate: current_term: 1 leader_uuid: "5d21973ace6d4f4b9134cc426b07de88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } health_report { overall_health: UNKNOWN } } }
I20260928 06:40:17.217203 11142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:17.213602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35572 (local address 127.8.151.193:45853)
0928 06:40:17.213871 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:17.213879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:17.213893 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:40:17.213943 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:17.213946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:17.213995 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:17.214106 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:17.214348 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:17.215192 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:17.216014 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:17.216365 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:17.216456 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:17.216735 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:17.216819 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:17.216879 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:17.217010 (+ 131us) server_negotiation.cc:299] Negotiation successful
0928 06:40:17.217057 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":102,"threads_started":1}
I20260928 06:40:17.225490 11142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:17.222737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:36359 (local address 127.8.151.193:45853)
0928 06:40:17.222833 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:17.222838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:17.222898 (+ 60us) server_negotiation.cc:407] Connection header received
0928 06:40:17.223018 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:17.223021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:17.223056 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:17.223107 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:17.223348 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:17.223834 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:17.224598 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:17.224797 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:17.224876 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:17.225065 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:17.225125 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:17.225178 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:17.225256 (+ 78us) server_negotiation.cc:299] Negotiation successful
0928 06:40:17.225343 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":42}
I20260928 06:40:17.225490 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:17.222598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:45853 (local address 127.8.151.193:36359)
0928 06:40:17.222814 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:17.222835 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:17.222933 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:17.223119 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:17.223122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:17.223128 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:17.223259 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:17.223264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:17.223929 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:17.223933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:17.224505 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:17.224512 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:17.224882 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:17.224885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:17.224956 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:17.224976 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:17.225188 (+ 212us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:17.225194 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:40:17.225249 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 06:40:17.225349 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":20,"thread_start_us":84,"threads_started":1}
I20260928 06:40:17.231047 10826 raft_consensus.cc:1261] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Refusing update from remote peer 5d21973ace6d4f4b9134cc426b07de88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:17.231047 10966 raft_consensus.cc:1261] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Refusing update from remote peer 5d21973ace6d4f4b9134cc426b07de88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:17.231436 11042 consensus_queue.cc:1035] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:17.231613 11102 consensus_queue.cc:1035] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:17.260171 10528 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:17.260332 10528 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:17.262306 10528 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07: Sending DeleteTablet for 3 replicas of tablet 549b0b6f774f40c38ec5d57837f69c4f
I20260928 06:40:17.262986 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet 549b0b6f774f40c38ec5d57837f69c4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:17 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:41990
I20260928 06:40:17.262992 10946 tablet_service.cc:1514] Processing DeleteTablet for tablet 549b0b6f774f40c38ec5d57837f69c4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:17 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48026
I20260928 06:40:17.263186 11123 tablet_replica.cc:330] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f: stopping tablet replica
I20260928 06:40:17.263193 11124 tablet_replica.cc:330] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26: stopping tablet replica
I20260928 06:40:17.263303 11123 raft_consensus.cc:2229] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:17.263317 11124 raft_consensus.cc:2229] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:17.263420 11124 raft_consensus.cc:2258] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:17.263427 11123 raft_consensus.cc:2258] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:17.264057 11124 ts_tablet_manager.cc:1905] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:17.264091 11123 ts_tablet_manager.cc:1905] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:17.266415 11123 ts_tablet_manager.cc:1918] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:17.266415 11124 ts_tablet_manager.cc:1918] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:17.266490 11124 log.cc:1196] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/549b0b6f774f40c38ec5d57837f69c4f
I20260928 06:40:17.266490 11123 log.cc:1196] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/549b0b6f774f40c38ec5d57837f69c4f
I20260928 06:40:17.266723 11124 ts_tablet_manager.cc:1939] T 549b0b6f774f40c38ec5d57837f69c4f P 10db4b9e0d7047f899f6e89c55b91e26: Deleting consensus metadata
I20260928 06:40:17.266723 11123 ts_tablet_manager.cc:1939] T 549b0b6f774f40c38ec5d57837f69c4f P e9cdbbc1620943d7b35e402f2427423f: Deleting consensus metadata
I20260928 06:40:17.267064 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet 549b0b6f774f40c38ec5d57837f69c4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:17 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:60226
I20260928 06:40:17.267261 11122 tablet_replica.cc:330] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88: stopping tablet replica
I20260928 06:40:17.267387 11122 raft_consensus.cc:2229] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:17.267549 10513 catalog_manager.cc:4841] TS e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655): tablet 549b0b6f774f40c38ec5d57837f69c4f (table test-table [id=c7b5508c4d8a4192baf1f73da57607d0]) successfully deleted
I20260928 06:40:17.267616 10516 catalog_manager.cc:4841] TS 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599): tablet 549b0b6f774f40c38ec5d57837f69c4f (table test-table [id=c7b5508c4d8a4192baf1f73da57607d0]) successfully deleted
I20260928 06:40:17.267696 11122 raft_consensus.cc:2258] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:17.267169 10528 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:17.268203 10528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:17.268349 11122 ts_tablet_manager.cc:1905] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:17.270759 11122 ts_tablet_manager.cc:1918] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:17.270865 11122 log.cc:1196] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/549b0b6f774f40c38ec5d57837f69c4f
I20260928 06:40:17.271098 11122 ts_tablet_manager.cc:1939] T 549b0b6f774f40c38ec5d57837f69c4f P 5d21973ace6d4f4b9134cc426b07de88: Deleting consensus metadata
I20260928 06:40:17.271879 10516 catalog_manager.cc:4841] TS 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853): tablet 549b0b6f774f40c38ec5d57837f69c4f (table test-table [id=c7b5508c4d8a4192baf1f73da57607d0]) successfully deleted
I20260928 06:40:17.272747 10665 tablet_service.cc:1467] Processing CreateTablet for tablet 49a704275744490c8a0bb865215cc860 (DEFAULT_TABLE table=test-table [id=218052fdfb324818834c3e7a5fdb844e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:17.272910 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 49a704275744490c8a0bb865215cc860 (DEFAULT_TABLE table=test-table [id=218052fdfb324818834c3e7a5fdb844e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:17.273010 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a704275744490c8a0bb865215cc860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:17.273015 10946 tablet_service.cc:1467] Processing CreateTablet for tablet 49a704275744490c8a0bb865215cc860 (DEFAULT_TABLE table=test-table [id=218052fdfb324818834c3e7a5fdb844e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:17.273142 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a704275744490c8a0bb865215cc860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:17.273231 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a704275744490c8a0bb865215cc860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:17.275401 11134 tablet_bootstrap.cc:492] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26: Bootstrap starting.
I20260928 06:40:17.275530 11133 tablet_bootstrap.cc:492] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f: Bootstrap starting.
I20260928 06:40:17.276279 11133 tablet_bootstrap.cc:654] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:17.276279 11134 tablet_bootstrap.cc:654] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:17.277117 11135 tablet_bootstrap.cc:492] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88: Bootstrap starting.
I20260928 06:40:17.277935 11135 tablet_bootstrap.cc:654] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:17.278707 11133 tablet_bootstrap.cc:492] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f: No bootstrap required, opened a new log
I20260928 06:40:17.278789 11133 ts_tablet_manager.cc:1397] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260928 06:40:17.278975 11135 tablet_bootstrap.cc:492] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88: No bootstrap required, opened a new log
I20260928 06:40:17.279062 11135 ts_tablet_manager.cc:1397] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:17.279565 11134 tablet_bootstrap.cc:492] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26: No bootstrap required, opened a new log
I20260928 06:40:17.279520 11135 raft_consensus.cc:348] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.279633 11135 raft_consensus.cc:374] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:17.279652 11134 ts_tablet_manager.cc:1397] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260928 06:40:17.279661 11135 raft_consensus.cc:729] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d21973ace6d4f4b9134cc426b07de88, State: Initialized, Role: FOLLOWER
I20260928 06:40:17.279775 11135 consensus_queue.cc:260] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.280033 11135 ts_tablet_manager.cc:1428] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:17.280042 11134 raft_consensus.cc:348] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.280150 11134 raft_consensus.cc:374] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:17.280182 11134 raft_consensus.cc:729] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10db4b9e0d7047f899f6e89c55b91e26, State: Initialized, Role: FOLLOWER
I20260928 06:40:17.280295 11134 consensus_queue.cc:260] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.280550 11134 ts_tablet_manager.cc:1428] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260928 06:40:17.280787 11133 raft_consensus.cc:348] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.280916 11133 raft_consensus.cc:374] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:17.280952 11133 raft_consensus.cc:729] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cdbbc1620943d7b35e402f2427423f, State: Initialized, Role: FOLLOWER
I20260928 06:40:17.281073 11133 consensus_queue.cc:260] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.281303 11133 ts_tablet_manager.cc:1428] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:17.336077 11139 raft_consensus.cc:482] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:17.336241 11139 raft_consensus.cc:504] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.336678 11139 leader_election.cc:288] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853), e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655)
I20260928 06:40:17.337144 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a704275744490c8a0bb865215cc860" candidate_uuid: "10db4b9e0d7047f899f6e89c55b91e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d21973ace6d4f4b9134cc426b07de88" is_pre_election: true
I20260928 06:40:17.337144 10826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a704275744490c8a0bb865215cc860" candidate_uuid: "10db4b9e0d7047f899f6e89c55b91e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cdbbc1620943d7b35e402f2427423f" is_pre_election: true
I20260928 06:40:17.337301 10826 raft_consensus.cc:2454] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10db4b9e0d7047f899f6e89c55b91e26 in term 0.
I20260928 06:40:17.337301 10686 raft_consensus.cc:2454] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10db4b9e0d7047f899f6e89c55b91e26 in term 0.
I20260928 06:40:17.337615 10900 leader_election.cc:302] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 10db4b9e0d7047f899f6e89c55b91e26, 5d21973ace6d4f4b9134cc426b07de88, e9cdbbc1620943d7b35e402f2427423f; no voters:
I20260928 06:40:17.337803 11139 raft_consensus.cc:2781] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:17.337860 11139 raft_consensus.cc:482] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:17.337891 11139 raft_consensus.cc:3037] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:17.339104 11139 raft_consensus.cc:504] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.339473 11139 leader_election.cc:288] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [CANDIDATE]: Term 1 election: Requested vote from peers 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853), e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655)
I20260928 06:40:17.339769 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a704275744490c8a0bb865215cc860" candidate_uuid: "10db4b9e0d7047f899f6e89c55b91e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d21973ace6d4f4b9134cc426b07de88"
I20260928 06:40:17.339797 10826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a704275744490c8a0bb865215cc860" candidate_uuid: "10db4b9e0d7047f899f6e89c55b91e26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cdbbc1620943d7b35e402f2427423f"
I20260928 06:40:17.339896 10686 raft_consensus.cc:3037] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:17.339905 10826 raft_consensus.cc:3037] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:17.340893 10826 raft_consensus.cc:2454] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10db4b9e0d7047f899f6e89c55b91e26 in term 1.
I20260928 06:40:17.340998 10686 raft_consensus.cc:2454] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10db4b9e0d7047f899f6e89c55b91e26 in term 1.
I20260928 06:40:17.341171 10897 leader_election.cc:302] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10db4b9e0d7047f899f6e89c55b91e26, e9cdbbc1620943d7b35e402f2427423f; no voters:
I20260928 06:40:17.341353 11139 raft_consensus.cc:2781] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:17.341526 11139 raft_consensus.cc:686] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 LEADER]: Becoming Leader. State: Replica: 10db4b9e0d7047f899f6e89c55b91e26, State: Running, Role: LEADER
I20260928 06:40:17.341663 11139 consensus_queue.cc:237] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } }
I20260928 06:40:17.343341 10529 catalog_manager.cc:5495] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195). New cstate: current_term: 1 leader_uuid: "10db4b9e0d7047f899f6e89c55b91e26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10db4b9e0d7047f899f6e89c55b91e26" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43599 } health_report { overall_health: HEALTHY } } }
I20260928 06:40:17.366261 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:17.363323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43116 (local address 127.8.151.195:43599)
0928 06:40:17.363614 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:17.363621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:17.363631 (+ 10us) server_negotiation.cc:407] Connection header received
0928 06:40:17.363664 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:17.363666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:17.363704 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:17.363772 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:17.364009 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:17.364528 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:17.365301 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:17.365522 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:17.365605 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:17.365844 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:17.365913 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:17.365968 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:17.366062 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 06:40:17.366112 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":122,"threads_started":1}
I20260928 06:40:17.373821 10826 raft_consensus.cc:1261] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Refusing update from remote peer 10db4b9e0d7047f899f6e89c55b91e26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:17.373889 10686 raft_consensus.cc:1261] T 49a704275744490c8a0bb865215cc860 P 5d21973ace6d4f4b9134cc426b07de88 [term 1 FOLLOWER]: Refusing update from remote peer 10db4b9e0d7047f899f6e89c55b91e26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:17.374300 11153 consensus_queue.cc:1035] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cdbbc1620943d7b35e402f2427423f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:17.374490 11139 consensus_queue.cc:1035] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d21973ace6d4f4b9134cc426b07de88" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 06:40:17.388365 11104 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 06:40:18.935606 10526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:18.935758 10526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:18.938175 10526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e570349406b34b0d99ca698b2eb6db07: Sending DeleteTablet for 3 replicas of tablet 49a704275744490c8a0bb865215cc860
I20260928 06:40:18.938719 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a704275744490c8a0bb865215cc860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:18 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:60226
I20260928 06:40:18.938769 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a704275744490c8a0bb865215cc860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:18 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:41990
I20260928 06:40:18.938747 10946 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a704275744490c8a0bb865215cc860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:18 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48026
I20260928 06:40:18.939213 11186 tablet_replica.cc:330] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f: stopping tablet replica
I20260928 06:40:18.939220 11187 tablet_replica.cc:330] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26: stopping tablet replica
I20260928 06:40:18.939357 11186 raft_consensus.cc:2229] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:18.939397 11187 raft_consensus.cc:2229] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:18.939515 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 10601
I20260928 06:40:18.940966 11186 raft_consensus.cc:2258] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:18.941529 11187 raft_consensus.cc:2258] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:18.941579 11186 ts_tablet_manager.cc:1905] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:18.942150 11187 ts_tablet_manager.cc:1905] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:18.943755 11186 ts_tablet_manager.cc:1918] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 06:40:18.943822 11186 log.cc:1196] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/49a704275744490c8a0bb865215cc860
I20260928 06:40:18.944036 11186 ts_tablet_manager.cc:1939] T 49a704275744490c8a0bb865215cc860 P e9cdbbc1620943d7b35e402f2427423f: Deleting consensus metadata
I20260928 06:40:18.944650 11187 ts_tablet_manager.cc:1918] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 06:40:18.944646 10513 catalog_manager.cc:4841] TS e9cdbbc1620943d7b35e402f2427423f (127.8.151.194:42655): tablet 49a704275744490c8a0bb865215cc860 (table test-table [id=218052fdfb324818834c3e7a5fdb844e]) successfully deleted
I20260928 06:40:18.944726 11187 log.cc:1196] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/49a704275744490c8a0bb865215cc860
I20260928 06:40:18.944943 11187 ts_tablet_manager.cc:1939] T 49a704275744490c8a0bb865215cc860 P 10db4b9e0d7047f899f6e89c55b91e26: Deleting consensus metadata
I20260928 06:40:18.945855 10516 catalog_manager.cc:4841] TS 10db4b9e0d7047f899f6e89c55b91e26 (127.8.151.195:43599): tablet 49a704275744490c8a0bb865215cc860 (table test-table [id=218052fdfb324818834c3e7a5fdb844e]) successfully deleted
W20260928 06:40:18.947341 10516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:45853 (error 108)
I20260928 06:40:18.947517 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 10741
I20260928 06:40:18.948980 11189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:18.948529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:45853 (local address 127.0.0.1:35582)
0928 06:40:18.948783 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:18.948877 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:45853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":81,"threads_started":1}
W20260928 06:40:18.949119 10516 catalog_manager.cc:4584] TS 5d21973ace6d4f4b9134cc426b07de88 (127.8.151.193:45853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49a704275744490c8a0bb865215cc860: Network error: Client connection negotiation failed: client connection to 127.8.151.193:45853: connect: Connection refused (error 111)
I20260928 06:40:18.954419 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 10881
I20260928 06:40:18.959892 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 10497
2026-09-28T06:40:18Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6011 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:19 dist-test-slave-gtr8 krb5kdc[11192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:19 dist-test-slave-gtr8 krb5kdc[11192](info): set up 2 sockets
Sep 28 06:40:19 dist-test-slave-gtr8 krb5kdc[11192](info): commencing operation
krb5kdc: starting...
W20260928 06:40:19.915405 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:19 dist-test-slave-gtr8 krb5kdc[11192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:20.033458 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:34577
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:39889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:34577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:20.138777 11205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:20.139057 11205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:20.139122 11205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:20.143122 11205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:20.143198 11205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:20.143215 11205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:20.143231 11205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:20.143244 11205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:20.148460 11205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:39889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:34577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:34577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:20.149438 11205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:20.150372 11205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:20.158938 11213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:20.159030 11205 server_base.cc:1034] running on GCE node
W20260928 06:40:20.158950 11210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:20.158953 11211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:20.159814 11205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:20.160876 11205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:20.162045 11205 hybrid_clock.cc:648] HybridClock initialized: now 1790577620162025 us; error 32 us; skew 500 ppm
Sep 28 06:40:20 dist-test-slave-gtr8 krb5kdc[11192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:20.168054 11205 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:20.169065 11205 webserver.cc:460] Webserver started at http://127.8.151.254:42755/ using document root <none> and password file <none>
I20260928 06:40:20.169620 11205 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:20.169685 11205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:20.169886 11205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:20.172021 11205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "a25f79c1fb324325b9c85121204d106a"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "db6713be3f6885f283a3af0611706030"
server_key_iv: "456e2b9b937860e57cc6a9feb5c4eea2"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.172559 11205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "a25f79c1fb324325b9c85121204d106a"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "db6713be3f6885f283a3af0611706030"
server_key_iv: "456e2b9b937860e57cc6a9feb5c4eea2"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.176275 11205 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260928 06:40:20.178653 11220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:20.179948 11205 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:20.180052 11205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "a25f79c1fb324325b9c85121204d106a"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "db6713be3f6885f283a3af0611706030"
server_key_iv: "456e2b9b937860e57cc6a9feb5c4eea2"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.180159 11205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:20.206691 11205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:20.209741 11205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:20.209896 11205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:20.216542 11205 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:34577
I20260928 06:40:20.216552 11272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:34577 every 8 connection(s)
I20260928 06:40:20.217455 11205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:20.219267 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 11205
I20260928 06:40:20.219370 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:20.219672 8799 external_mini_cluster.cc:1428] Setting key f14d39941542afd8a989852c3b5a4a1a
I20260928 06:40:20.220777 11273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:20.226505 11273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a: Bootstrap starting.
Sep 28 06:40:20 dist-test-slave-gtr8 krb5kdc[11192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:20.228730 11273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:20.229417 11273 log.cc:824] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:20.231351 11273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a: No bootstrap required, opened a new log
I20260928 06:40:20.233186 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:20.221161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58798 (local address 127.8.151.254:34577)
0928 06:40:20.221629 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:20.221645 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:20.221682 (+ 37us) server_negotiation.cc:407] Connection header received
0928 06:40:20.222351 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:20.222368 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:20.222689 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:20.223015 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:20.223879 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.224883 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.225775 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.226261 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.226401 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:20.228748 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:20.228753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:20.228766 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:20.228793 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:20.230529 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:20.230929 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:20.230935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:20.230941 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:20.231050 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:20.231327 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:20.231331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:20.231332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:20.231745 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:20.231935 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:20.232200 (+ 265us) server_negotiation.cc:299] Negotiation successful
0928 06:40:20.232476 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":157,"threads_started":1}
I20260928 06:40:20.234833 11273 raft_consensus.cc:348] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25f79c1fb324325b9c85121204d106a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34577 } }
I20260928 06:40:20.235051 11273 raft_consensus.cc:374] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:20.235103 11273 raft_consensus.cc:729] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a25f79c1fb324325b9c85121204d106a, State: Initialized, Role: FOLLOWER
I20260928 06:40:20.235519 11273 consensus_queue.cc:260] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25f79c1fb324325b9c85121204d106a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34577 } }
I20260928 06:40:20.235646 11273 raft_consensus.cc:388] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:20.235705 11273 raft_consensus.cc:482] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:20.235798 11273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:20.236616 11273 raft_consensus.cc:504] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25f79c1fb324325b9c85121204d106a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34577 } }
I20260928 06:40:20.236922 11273 leader_election.cc:302] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a25f79c1fb324325b9c85121204d106a; no voters:
I20260928 06:40:20.237185 11273 leader_election.cc:288] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:20.237320 11278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:20.237564 11278 raft_consensus.cc:686] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [term 1 LEADER]: Becoming Leader. State: Replica: a25f79c1fb324325b9c85121204d106a, State: Running, Role: LEADER
I20260928 06:40:20.237901 11278 consensus_queue.cc:237] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25f79c1fb324325b9c85121204d106a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34577 } }
I20260928 06:40:20.238304 11273 sys_catalog.cc:564] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:20.239395 11279 sys_catalog.cc:455] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a25f79c1fb324325b9c85121204d106a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25f79c1fb324325b9c85121204d106a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34577 } } }
I20260928 06:40:20.239507 11279 sys_catalog.cc:458] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:20.239894 11287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:20.239846 11280 sys_catalog.cc:455] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a25f79c1fb324325b9c85121204d106a. Latest consensus state: current_term: 1 leader_uuid: "a25f79c1fb324325b9c85121204d106a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25f79c1fb324325b9c85121204d106a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34577 } } }
I20260928 06:40:20.240010 11280 sys_catalog.cc:458] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:20.242797 11287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:20.248821 11287 catalog_manager.cc:1347] Generated new cluster ID: 5c6ca8c34ea545ff961835fe24c2a546
I20260928 06:40:20.248886 11287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:20.257131 11287 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:20.258327 11287 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:20.270161 11287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a25f79c1fb324325b9c85121204d106a: Generated new TSK 0
I20260928 06:40:20.270967 11287 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:40:20.302386 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34577
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:39889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:20.408423 11299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:20.408671 11299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:20.408730 11299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:20.412626 11299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:20.412675 11299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:20.412750 11299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:20.417758 11299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:39889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:20.418707 11299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:20.419728 11299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:20.428849 11305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:20.428823 11307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:20.428835 11304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:20.429049 11299 server_base.cc:1034] running on GCE node
I20260928 06:40:20.429589 11299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:20.430233 11299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:20.431384 11299 hybrid_clock.cc:648] HybridClock initialized: now 1790577620431375 us; error 30 us; skew 500 ppm
Sep 28 06:40:20 dist-test-slave-gtr8 krb5kdc[11192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:20.434042 11299 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:40:20.434840 11299 webserver.cc:460] Webserver started at http://127.8.151.193:46837/ using document root <none> and password file <none>
I20260928 06:40:20.435397 11299 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:20.435465 11299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:20.435647 11299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:20.437572 11299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "9a04848dcc9343f4aa24e919bb0f62e5"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "7f58ce990d378cbad1afff4acd05b8e0"
server_key_iv: "193953dd7e07a9d942b8a1b48129d3ab"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.438084 11299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "9a04848dcc9343f4aa24e919bb0f62e5"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "7f58ce990d378cbad1afff4acd05b8e0"
server_key_iv: "193953dd7e07a9d942b8a1b48129d3ab"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.441884 11299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:40:20.444322 11314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:20.445621 11299 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:20.445719 11299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "9a04848dcc9343f4aa24e919bb0f62e5"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "7f58ce990d378cbad1afff4acd05b8e0"
server_key_iv: "193953dd7e07a9d942b8a1b48129d3ab"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.445832 11299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:20.465395 11299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:20.468314 11299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:20.468484 11299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:20.469051 11299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:20.469817 11299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:20.469861 11299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:20.469902 11299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:20.469938 11299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:20.478474 11299 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:34969
I20260928 06:40:20.478502 11427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:34969 every 8 connection(s)
I20260928 06:40:20.479367 11299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
Sep 28 06:40:20 dist-test-slave-gtr8 krb5kdc[11192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:20.488678 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 11299
I20260928 06:40:20.488791 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:20.489090 8799 external_mini_cluster.cc:1428] Setting key 5572e4b3271da690fb85d560e72f92ca
I20260928 06:40:20.492614 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:20.481125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:56753 (local address 127.8.151.254:34577)
0928 06:40:20.481280 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:20.481285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:20.482303 (+ 1018us) server_negotiation.cc:407] Connection header received
0928 06:40:20.483330 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:20.483333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:20.483381 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:20.483446 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:20.484679 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.485229 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.486325 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.486568 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.486652 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:20.489273 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:20.489277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:20.489279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:20.489304 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:20.490604 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:20.491133 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:20.491163 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:20.491165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:20.491241 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:20.491579 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:20.491581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:20.491582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:20.491742 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:20.491825 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:20.492319 (+ 494us) server_negotiation.cc:299] Negotiation successful
0928 06:40:20.492457 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 06:40:20.493597 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:20.481520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:34577 (local address 127.8.151.193:56753)
0928 06:40:20.482164 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:20.482205 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:20.483078 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:20.483594 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:20.483601 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:20.483942 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:20.484508 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:20.484525 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.485355 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.485358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:20.486201 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:20.486210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.486728 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.486732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:20.486873 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:20.487457 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:20.487473 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:20.489112 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:20.490709 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:20.490714 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:20.490730 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:20.491050 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:20.491318 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:20.491321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:20.491322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:20.491499 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:20.491855 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:20.491861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:20.492096 (+ 235us) client_negotiation.cc:769] Sending connection context
0928 06:40:20.492262 (+ 166us) client_negotiation.cc:240] Negotiation successful
0928 06:40:20.492582 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":115,"threads_started":1}
I20260928 06:40:20.494730 11428 heartbeater.cc:347] Connected to a master server at 127.8.151.254:34577
I20260928 06:40:20.494961 11428 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:20.495474 11428 heartbeater.cc:511] Master 127.8.151.254:34577 requested a full tablet report, sending...
I20260928 06:40:20.496879 11234 ts_manager.cc:194] Registered new tserver with Master: 9a04848dcc9343f4aa24e919bb0f62e5 (127.8.151.193:34969)
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:40:20.498019 11234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:56753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:40:20.514218 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34577
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:39889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:20.619663 11433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:20.619899 11433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:20.619939 11433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:20.623903 11433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:20.623956 11433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:20.624015 11433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:20.629024 11433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:39889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:20.629987 11433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:20.630925 11433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:20.640153 11438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:20.640161 11439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:20.640184 11441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:20.640333 11433 server_base.cc:1034] running on GCE node
I20260928 06:40:20.640836 11433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:20.641449 11433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:20.642637 11433 hybrid_clock.cc:648] HybridClock initialized: now 1790577620642610 us; error 37 us; skew 500 ppm
Sep 28 06:40:20 dist-test-slave-gtr8 krb5kdc[11192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:20.645345 11433 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:40:20.646252 11433 webserver.cc:460] Webserver started at http://127.8.151.194:38921/ using document root <none> and password file <none>
I20260928 06:40:20.646803 11433 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:20.646874 11433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:20.647078 11433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:20.649313 11433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "225da7fcb6bf43378b8a2e38ae684d37"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "9e2ec8e873ef1eae4de50f5cc459a32c"
server_key_iv: "123024b333f8e38bc86f90d2e8fb4fa0"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.649883 11433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "225da7fcb6bf43378b8a2e38ae684d37"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "9e2ec8e873ef1eae4de50f5cc459a32c"
server_key_iv: "123024b333f8e38bc86f90d2e8fb4fa0"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.653606 11433 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260928 06:40:20.656139 11448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:20.657356 11433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:20.657449 11433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "225da7fcb6bf43378b8a2e38ae684d37"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "9e2ec8e873ef1eae4de50f5cc459a32c"
server_key_iv: "123024b333f8e38bc86f90d2e8fb4fa0"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.657557 11433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:20.675859 11433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:20.679028 11433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:20.679243 11433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:20.679917 11433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:20.680879 11433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:20.680931 11433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:20.680979 11433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:20.681051 11433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:20.690359 11433 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:35351
I20260928 06:40:20.690403 11561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:35351 every 8 connection(s)
I20260928 06:40:20.691296 11433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:20.699978 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 11433
I20260928 06:40:20.700069 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
Sep 28 06:40:20 dist-test-slave-gtr8 krb5kdc[11192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:20.700325 8799 external_mini_cluster.cc:1428] Setting key b404e2c259c5348467cf2576ee738906
I20260928 06:40:20.704322 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:20.693096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:39895 (local address 127.8.151.254:34577)
0928 06:40:20.693235 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:20.693239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:20.694249 (+ 1010us) server_negotiation.cc:407] Connection header received
0928 06:40:20.695162 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:20.695166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:20.695223 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:20.695297 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:20.696425 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.696999 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.698112 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.698335 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.698421 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:20.700808 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:20.700812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:20.700815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:20.700853 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:20.702169 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:20.702716 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:20.702718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:20.702719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:20.702824 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:20.703246 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:20.703249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:20.703250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:20.703468 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:20.703570 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:20.704042 (+ 472us) server_negotiation.cc:299] Negotiation successful
0928 06:40:20.704164 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:40:20.705175 11564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:20.693469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:34577 (local address 127.8.151.194:39895)
0928 06:40:20.694105 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:20.694145 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:20.694942 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:20.695483 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:20.695490 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:20.695829 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:20.696280 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:20.696294 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.697118 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.697120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:20.697992 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:20.698001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.698475 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.698479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:20.698635 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:20.699199 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:20.699215 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:20.700645 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:20.702281 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:20.702287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:20.702301 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:20.702624 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:20.702949 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:20.702952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:20.702954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:20.703127 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:20.703579 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:20.703584 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:20.703813 (+ 229us) client_negotiation.cc:769] Sending connection context
0928 06:40:20.703977 (+ 164us) client_negotiation.cc:240] Negotiation successful
0928 06:40:20.704224 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":115,"threads_started":1}
I20260928 06:40:20.706411 11562 heartbeater.cc:347] Connected to a master server at 127.8.151.254:34577
I20260928 06:40:20.706622 11562 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:20.707049 11562 heartbeater.cc:511] Master 127.8.151.254:34577 requested a full tablet report, sending...
I20260928 06:40:20.708019 11234 ts_manager.cc:194] Registered new tserver with Master: 225da7fcb6bf43378b8a2e38ae684d37 (127.8.151.194:35351)
I20260928 06:40:20.708551 11234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:39895
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:40:20.730144 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34577
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:39889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:20.841712 11567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:20.841995 11567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:20.842039 11567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:20.846127 11567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:20.846184 11567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:20.846244 11567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:20.851589 11567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:39889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:34577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:20.852610 11567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:20.853587 11567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:20.864195 11575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:20.864219 11572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:20.864189 11573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:20.864282 11567 server_base.cc:1034] running on GCE node
I20260928 06:40:20.864847 11567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:20.865542 11567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:20.866731 11567 hybrid_clock.cc:648] HybridClock initialized: now 1790577620866706 us; error 32 us; skew 500 ppm
Sep 28 06:40:20 dist-test-slave-gtr8 krb5kdc[11192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:20.869468 11567 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:40:20.870312 11567 webserver.cc:460] Webserver started at http://127.8.151.195:35763/ using document root <none> and password file <none>
I20260928 06:40:20.870843 11567 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:20.870900 11567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:20.871070 11567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:20.873178 11567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "a2a7a0517df743da98653cb470d12596"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "a2ee1e7a337def49b13153a972619042"
server_key_iv: "fc0aeaf8ce6f7264337b102eed3a794b"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.873684 11567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "a2a7a0517df743da98653cb470d12596"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "a2ee1e7a337def49b13153a972619042"
server_key_iv: "fc0aeaf8ce6f7264337b102eed3a794b"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.877573 11567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:40:20.880093 11582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:20.881376 11567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:20.881477 11567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "a2a7a0517df743da98653cb470d12596"
format_stamp: "Formatted at 2026-09-28 06:40:20 on dist-test-slave-gtr8"
server_key: "a2ee1e7a337def49b13153a972619042"
server_key_iv: "fc0aeaf8ce6f7264337b102eed3a794b"
server_key_version: "encryptionkey@0"
I20260928 06:40:20.881568 11567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:20.894325 11567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:20.897645 11567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:20.897796 11567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:20.898334 11567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:20.899116 11567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:20.899192 11567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:20.899243 11567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:20.899266 11567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:20.907395 11567 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:37031
I20260928 06:40:20.907438 11695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:37031 every 8 connection(s)
I20260928 06:40:20.908269 11567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:20.916455 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 11567
I20260928 06:40:20.916561 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:20.916818 8799 external_mini_cluster.cc:1428] Setting key 88c434501957c5639b1b7983584bba68
Sep 28 06:40:20 dist-test-slave-gtr8 krb5kdc[11192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:20.921674 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:20.910062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:33555 (local address 127.8.151.254:34577)
0928 06:40:20.910201 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:20.910205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:20.911358 (+ 1153us) server_negotiation.cc:407] Connection header received
0928 06:40:20.912232 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:20.912236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:20.912292 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:20.912376 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:20.913553 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.914069 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.915196 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.915422 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.915509 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:20.917983 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:20.917987 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:20.917989 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:20.918018 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:20.919569 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:20.920105 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:20.920108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:20.920109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:20.920209 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:20.920536 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:20.920538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:20.920539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:20.920749 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:20.920852 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:20.921382 (+ 530us) server_negotiation.cc:299] Negotiation successful
0928 06:40:20.921510 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 06:40:20.922544 11698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:20.910475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:34577 (local address 127.8.151.195:33555)
0928 06:40:20.911201 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:20.911248 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:20.912032 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:20.912554 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:20.912561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:20.912948 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:20.913413 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:20.913425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.914180 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.914183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:20.915072 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:20.915081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:20.915561 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:20.915565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:20.915723 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:20.916291 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:20.916307 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:20.917813 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:20.919688 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:20.919693 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:20.919707 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:20.919988 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:20.920319 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:20.920321 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:20.920322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:20.920456 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:20.920849 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:20.920854 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:20.921097 (+ 243us) client_negotiation.cc:769] Sending connection context
0928 06:40:20.921293 (+ 196us) client_negotiation.cc:240] Negotiation successful
0928 06:40:20.921649 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":113,"threads_started":1}
I20260928 06:40:20.923693 11696 heartbeater.cc:347] Connected to a master server at 127.8.151.254:34577
I20260928 06:40:20.923964 11696 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:20.924453 11696 heartbeater.cc:511] Master 127.8.151.254:34577 requested a full tablet report, sending...
I20260928 06:40:20.925444 11234 ts_manager.cc:194] Registered new tserver with Master: a2a7a0517df743da98653cb470d12596 (127.8.151.195:37031)
I20260928 06:40:20.926021 11234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:33555
I20260928 06:40:20.930608 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 06:40:20.942916 11276 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:20.941385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58808 (local address 127.8.151.254:34577)
0928 06:40:20.941509 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:20.941513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:20.941691 (+ 178us) server_negotiation.cc:407] Connection header received
0928 06:40:20.941784 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:20.941787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:20.941872 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:20.941953 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:20.942802 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58808: BlockingRecv error: recv got EOF from 127.0.0.1:58808 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 06:40:20.944751 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 11299
I20260928 06:40:20.949692 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 11433
I20260928 06:40:20.954272 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 11567
I20260928 06:40:20.958642 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 11205
2026-09-28T06:40:20Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1997 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:21 dist-test-slave-gtr8 krb5kdc[11708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:21 dist-test-slave-gtr8 krb5kdc[11708](info): set up 2 sockets
Sep 28 06:40:21 dist-test-slave-gtr8 krb5kdc[11708](info): commencing operation
krb5kdc: starting...
W20260928 06:40:21.911383 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:21 dist-test-slave-gtr8 krb5kdc[11708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:22.028298 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39503
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:39503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:22.134231 11721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:22.134523 11721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:22.134599 11721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:22.138703 11721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:22.138768 11721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:22.138794 11721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:22.138815 11721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:22.138844 11721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:22.144474 11721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:39503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:22.145515 11721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:22.146512 11721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:22.154670 11726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:22.154670 11729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:22.154706 11721 server_base.cc:1034] running on GCE node
W20260928 06:40:22.154733 11727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:22.155507 11721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:22.156642 11721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:22.157828 11721 hybrid_clock.cc:648] HybridClock initialized: now 1790577622157799 us; error 39 us; skew 500 ppm
Sep 28 06:40:22 dist-test-slave-gtr8 krb5kdc[11708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:22.160552 11721 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:22.161381 11721 webserver.cc:460] Webserver started at http://127.8.151.254:41039/ using document root <none> and password file <none>
I20260928 06:40:22.161891 11721 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:22.162002 11721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:22.162240 11721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:22.164479 11721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "5d65f442f3104d959cf859a8a240b3e3"
format_stamp: "Formatted at 2026-09-28 06:40:22 on dist-test-slave-gtr8"
server_key: "424509282959c371ae40c238c7c8fe2c"
server_key_iv: "03b959a851bf8ef524265e59b1c4f113"
server_key_version: "encryptionkey@0"
I20260928 06:40:22.165001 11721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "5d65f442f3104d959cf859a8a240b3e3"
format_stamp: "Formatted at 2026-09-28 06:40:22 on dist-test-slave-gtr8"
server_key: "424509282959c371ae40c238c7c8fe2c"
server_key_iv: "03b959a851bf8ef524265e59b1c4f113"
server_key_version: "encryptionkey@0"
I20260928 06:40:22.168524 11721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260928 06:40:22.170800 11736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:22.172029 11721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:22.172127 11721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "5d65f442f3104d959cf859a8a240b3e3"
format_stamp: "Formatted at 2026-09-28 06:40:22 on dist-test-slave-gtr8"
server_key: "424509282959c371ae40c238c7c8fe2c"
server_key_iv: "03b959a851bf8ef524265e59b1c4f113"
server_key_version: "encryptionkey@0"
I20260928 06:40:22.172224 11721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:22.188089 11721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:22.191226 11721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:22.191401 11721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:22.198136 11721 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:39503
I20260928 06:40:22.198395 11788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:39503 every 8 connection(s)
I20260928 06:40:22.199064 11721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:22.201812 11789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:22.204567 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 11721
I20260928 06:40:22.204667 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:22.204914 8799 external_mini_cluster.cc:1428] Setting key 686f23020373e95b846ae812ede2d406
I20260928 06:40:22.208061 11789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3: Bootstrap starting.
I20260928 06:40:22.210747 11789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:22.211530 11789 log.cc:824] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:22.213553 11789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3: No bootstrap required, opened a new log
Sep 28 06:40:22 dist-test-slave-gtr8 krb5kdc[11708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:22.217864 11789 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d65f442f3104d959cf859a8a240b3e3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39503 } }
I20260928 06:40:22.218143 11789 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:22.218209 11789 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d65f442f3104d959cf859a8a240b3e3, State: Initialized, Role: FOLLOWER
I20260928 06:40:22.218696 11789 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d65f442f3104d959cf859a8a240b3e3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39503 } }
I20260928 06:40:22.218840 11789 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:22.218912 11789 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:22.219027 11789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:22.219570 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:22.206228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.8.151.254:39503)
0928 06:40:22.206667 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:22.206688 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:22.206733 (+ 45us) server_negotiation.cc:407] Connection header received
0928 06:40:22.207583 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:22.207605 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:22.208019 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:22.208383 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:22.209010 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:22.210487 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:22.211438 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:22.212083 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:22.212209 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:22.214531 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:22.214537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:22.214555 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:22.214600 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:22.216625 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:22.217035 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:22.217042 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:22.217048 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:22.217196 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:22.217423 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:22.217427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:22.217429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:22.217942 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:22.218162 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:22.218391 (+ 229us) server_negotiation.cc:299] Negotiation successful
0928 06:40:22.218670 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":104,"threads_started":1}
I20260928 06:40:22.220009 11789 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d65f442f3104d959cf859a8a240b3e3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39503 } }
I20260928 06:40:22.220342 11789 leader_election.cc:302] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d65f442f3104d959cf859a8a240b3e3; no voters:
I20260928 06:40:22.220656 11789 leader_election.cc:288] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:22.220732 11794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:22.220968 11794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [term 1 LEADER]: Becoming Leader. State: Replica: 5d65f442f3104d959cf859a8a240b3e3, State: Running, Role: LEADER
I20260928 06:40:22.221264 11794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d65f442f3104d959cf859a8a240b3e3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39503 } }
I20260928 06:40:22.221966 11789 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:22.222419 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:40:22.222462 11796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d65f442f3104d959cf859a8a240b3e3. Latest consensus state: current_term: 1 leader_uuid: "5d65f442f3104d959cf859a8a240b3e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d65f442f3104d959cf859a8a240b3e3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39503 } } }
I20260928 06:40:22.222579 11796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:22.223255 11795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d65f442f3104d959cf859a8a240b3e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d65f442f3104d959cf859a8a240b3e3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39503 } } }
I20260928 06:40:22.223349 11795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3 [sys.catalog]: This master's current role is: LEADER
W20260928 06:40:22.227044 11811 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:40:22.227125 11811 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:40:22.227226 11799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:22.228144 11799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:22.234575 11799 catalog_manager.cc:1347] Generated new cluster ID: eff9ffd34d014ea59101058fceddc15a
I20260928 06:40:22.234644 11799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:22.243251 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:22.238808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39194 (local address 127.8.151.254:39503)
0928 06:40:22.238977 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:22.238982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:22.239173 (+ 191us) server_negotiation.cc:407] Connection header received
0928 06:40:22.239317 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:22.239320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:22.239360 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:22.239430 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:22.239937 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:22.240471 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:22.241292 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:22.241531 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:22.241619 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:22.242372 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:22.242376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:22.242379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:22.242793 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:22.242822 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:22.242837 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:22.242925 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:22.243019 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 06:40:22.243086 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 06:40:22.253638 11799 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:22.254602 11799 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:22.259090 11799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d65f442f3104d959cf859a8a240b3e3: Generated new TSK 0
I20260928 06:40:22.259765 11799 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:40:22.266191 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 11721
2026-09-28T06:40:22Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1306 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:22 dist-test-slave-gtr8 krb5kdc[11822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:22 dist-test-slave-gtr8 krb5kdc[11822](info): set up 2 sockets
Sep 28 06:40:22 dist-test-slave-gtr8 krb5kdc[11822](info): commencing operation
krb5kdc: starting...
W20260928 06:40:23.219444 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:23 dist-test-slave-gtr8 krb5kdc[11822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:23.335471 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:44145
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:44145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:23.454758 11835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:23.455024 11835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:23.455068 11835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:23.459007 11835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:23.459067 11835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:23.459086 11835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:23.459113 11835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:23.459127 11835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:23.464404 11835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:44145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:44145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:23.465384 11835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:23.466358 11835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:23.474499 11841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:23.474436 11840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:23.474433 11843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:23.474722 11835 server_base.cc:1034] running on GCE node
I20260928 06:40:23.475373 11835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:23.476392 11835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:23.477540 11835 hybrid_clock.cc:648] HybridClock initialized: now 1790577623477526 us; error 28 us; skew 500 ppm
Sep 28 06:40:23 dist-test-slave-gtr8 krb5kdc[11822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:23.480175 11835 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:23.480983 11835 webserver.cc:460] Webserver started at http://127.8.151.254:39481/ using document root <none> and password file <none>
I20260928 06:40:23.481474 11835 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:23.481534 11835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:23.481709 11835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:23.483706 11835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "00ca0c82fe944f9a878cf29ee6790313"
format_stamp: "Formatted at 2026-09-28 06:40:23 on dist-test-slave-gtr8"
server_key: "d302808a7d824bc59b675e919b77bece"
server_key_iv: "e6d5133ed68e7051759923218ffc17e6"
server_key_version: "encryptionkey@0"
I20260928 06:40:23.484215 11835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "00ca0c82fe944f9a878cf29ee6790313"
format_stamp: "Formatted at 2026-09-28 06:40:23 on dist-test-slave-gtr8"
server_key: "d302808a7d824bc59b675e919b77bece"
server_key_iv: "e6d5133ed68e7051759923218ffc17e6"
server_key_version: "encryptionkey@0"
I20260928 06:40:23.487664 11835 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:23.489830 11850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:23.490957 11835 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:23.491068 11835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "00ca0c82fe944f9a878cf29ee6790313"
format_stamp: "Formatted at 2026-09-28 06:40:23 on dist-test-slave-gtr8"
server_key: "d302808a7d824bc59b675e919b77bece"
server_key_iv: "e6d5133ed68e7051759923218ffc17e6"
server_key_version: "encryptionkey@0"
I20260928 06:40:23.491178 11835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:23.504135 11835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:23.507483 11835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:23.507658 11835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:23.514145 11835 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:44145
I20260928 06:40:23.514451 11902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:44145 every 8 connection(s)
I20260928 06:40:23.515080 11835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:23.517663 11903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:23.521039 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 11835
I20260928 06:40:23.521145 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:23.521378 8799 external_mini_cluster.cc:1428] Setting key f928aaa057a861efb14d74bbb15d94e4
I20260928 06:40:23.523010 11903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313: Bootstrap starting.
I20260928 06:40:23.525563 11903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:23.526373 11903 log.cc:824] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:23.528321 11903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313: No bootstrap required, opened a new log
Sep 28 06:40:23 dist-test-slave-gtr8 krb5kdc[11822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:23.532007 11903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca0c82fe944f9a878cf29ee6790313" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44145 } }
I20260928 06:40:23.532213 11903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:23.532276 11903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ca0c82fe944f9a878cf29ee6790313, State: Initialized, Role: FOLLOWER
I20260928 06:40:23.532677 11903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca0c82fe944f9a878cf29ee6790313" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44145 } }
I20260928 06:40:23.532792 11903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:23.532848 11903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:23.532941 11903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:23.533869 11903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca0c82fe944f9a878cf29ee6790313" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44145 } }
I20260928 06:40:23.534185 11903 leader_election.cc:302] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ca0c82fe944f9a878cf29ee6790313; no voters:
I20260928 06:40:23.534475 11903 leader_election.cc:288] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:23.534766 11908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:23.534945 11908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [term 1 LEADER]: Becoming Leader. State: Replica: 00ca0c82fe944f9a878cf29ee6790313, State: Running, Role: LEADER
I20260928 06:40:23.535270 11908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca0c82fe944f9a878cf29ee6790313" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44145 } }
I20260928 06:40:23.535650 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:23.522682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.8.151.254:44145)
0928 06:40:23.523089 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:23.523109 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:23.523178 (+ 69us) server_negotiation.cc:407] Connection header received
0928 06:40:23.523965 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:23.523988 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:23.524394 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:23.524731 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:23.525381 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:23.526809 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:23.527743 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:23.528427 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:23.528582 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:23.531040 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:23.531045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:23.531057 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:23.531081 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:23.532750 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:23.533266 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:23.533271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:23.533275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:23.533379 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:23.533632 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:23.533634 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:23.533635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:23.533992 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:23.534185 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:23.534411 (+ 226us) server_negotiation.cc:299] Negotiation successful
0928 06:40:23.534701 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":103,"threads_started":1}
I20260928 06:40:23.535766 11903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:23.536448 11909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ca0c82fe944f9a878cf29ee6790313. Latest consensus state: current_term: 1 leader_uuid: "00ca0c82fe944f9a878cf29ee6790313" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca0c82fe944f9a878cf29ee6790313" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44145 } } }
I20260928 06:40:23.536563 11909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:23.536932 11916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:23.536911 11910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ca0c82fe944f9a878cf29ee6790313" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ca0c82fe944f9a878cf29ee6790313" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44145 } } }
I20260928 06:40:23.537000 11910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:23.539923 11916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:23.545217 11916 catalog_manager.cc:1347] Generated new cluster ID: e79340ecc4ba4efb9ea2eb2683c1a8fe
I20260928 06:40:23.545281 11916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:23.565498 11916 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:23.566840 11916 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:23.581059 11916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ca0c82fe944f9a878cf29ee6790313: Generated new TSK 0
I20260928 06:40:23.581897 11916 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:40:23.614120 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:44145
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:23.721774 11929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:23.722030 11929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:23.722080 11929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:23.726076 11929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:23.726137 11929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:23.726197 11929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:23.731469 11929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:44145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:23.732478 11929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:23.733472 11929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:23.743005 11934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:23.742978 11935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:23.743127 11929 server_base.cc:1034] running on GCE node
W20260928 06:40:23.742978 11937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:23.743691 11929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:23.744293 11929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:23.745461 11929 hybrid_clock.cc:648] HybridClock initialized: now 1790577623745442 us; error 30 us; skew 500 ppm
Sep 28 06:40:23 dist-test-slave-gtr8 krb5kdc[11822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:23.748140 11929 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:40:23.748977 11929 webserver.cc:460] Webserver started at http://127.8.151.193:44441/ using document root <none> and password file <none>
I20260928 06:40:23.749488 11929 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:23.749570 11929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:23.749782 11929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:23.751892 11929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "88b86a33afd748a48601318306df4940"
format_stamp: "Formatted at 2026-09-28 06:40:23 on dist-test-slave-gtr8"
server_key: "3b7808255baef9eac4891a33d05198db"
server_key_iv: "2744ccb10a08727b1999832749130ade"
server_key_version: "encryptionkey@0"
I20260928 06:40:23.752404 11929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "88b86a33afd748a48601318306df4940"
format_stamp: "Formatted at 2026-09-28 06:40:23 on dist-test-slave-gtr8"
server_key: "3b7808255baef9eac4891a33d05198db"
server_key_iv: "2744ccb10a08727b1999832749130ade"
server_key_version: "encryptionkey@0"
I20260928 06:40:23.755930 11929 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:40:23.758170 11944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:23.759358 11929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:40:23.759466 11929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "88b86a33afd748a48601318306df4940"
format_stamp: "Formatted at 2026-09-28 06:40:23 on dist-test-slave-gtr8"
server_key: "3b7808255baef9eac4891a33d05198db"
server_key_iv: "2744ccb10a08727b1999832749130ade"
server_key_version: "encryptionkey@0"
I20260928 06:40:23.759567 11929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:23.777957 11929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:23.781104 11929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:23.781277 11929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:23.781862 11929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:23.782723 11929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:23.782770 11929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:23.782825 11929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:23.782864 11929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:23.791443 11929 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:32935
I20260928 06:40:23.791492 12057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:32935 every 8 connection(s)
I20260928 06:40:23.792309 11929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:23.801025 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 11929
I20260928 06:40:23.801128 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:23.801380 8799 external_mini_cluster.cc:1428] Setting key 1152220f7184d3c0eea33019fa7bb2f1
Sep 28 06:40:23 dist-test-slave-gtr8 krb5kdc[11822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:23.805933 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:23.794102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:52371 (local address 127.8.151.254:44145)
0928 06:40:23.794266 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:23.794270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:23.795169 (+ 899us) server_negotiation.cc:407] Connection header received
0928 06:40:23.796555 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:23.796559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:23.796615 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:23.796685 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:23.797852 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:23.798385 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:23.799562 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:23.799788 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:23.799877 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:23.802337 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:23.802340 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:23.802343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:23.802368 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:23.803947 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:23.804465 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:23.804468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:23.804470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:23.804561 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:23.804884 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:23.804888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:23.804890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:23.805092 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:23.805180 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:23.805708 (+ 528us) server_negotiation.cc:299] Negotiation successful
0928 06:40:23.805810 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 06:40:23.807040 12060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:23.794498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:44145 (local address 127.8.151.193:52371)
0928 06:40:23.795000 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:23.795043 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:23.796342 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:23.796868 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:23.796876 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:23.797233 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:23.797706 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:23.797719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:23.798505 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:23.798509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:23.799450 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:23.799459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:23.799931 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:23.799934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:23.800102 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:23.800649 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:23.800666 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:23.802168 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:23.804081 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:23.804086 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:23.804099 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:23.804368 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:23.804646 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:23.804649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:23.804651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:23.804788 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:23.805196 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:23.805203 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:23.805443 (+ 240us) client_negotiation.cc:769] Sending connection context
0928 06:40:23.805659 (+ 216us) client_negotiation.cc:240] Negotiation successful
0928 06:40:23.805993 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":100,"threads_started":1}
I20260928 06:40:23.808269 12058 heartbeater.cc:347] Connected to a master server at 127.8.151.254:44145
I20260928 06:40:23.808534 12058 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:23.809119 12058 heartbeater.cc:511] Master 127.8.151.254:44145 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:40:23.810662 11867 ts_manager.cc:194] Registered new tserver with Master: 88b86a33afd748a48601318306df4940 (127.8.151.193:32935)
I20260928 06:40:23.811811 11867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:52371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:40:23.828444 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:44145
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:23.938341 12063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:23.938601 12063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:23.938653 12063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:23.942706 12063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:23.942773 12063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:23.942847 12063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:23.948170 12063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:44145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:23.949252 12063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:23.950253 12063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:23.959889 12068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:23.960031 12063 server_base.cc:1034] running on GCE node
W20260928 06:40:23.959897 12069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:23.959889 12071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:23.960642 12063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:23.961273 12063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:23.962435 12063 hybrid_clock.cc:648] HybridClock initialized: now 1790577623962416 us; error 30 us; skew 500 ppm
Sep 28 06:40:23 dist-test-slave-gtr8 krb5kdc[11822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:23.965114 12063 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:40:23.965948 12063 webserver.cc:460] Webserver started at http://127.8.151.194:35057/ using document root <none> and password file <none>
I20260928 06:40:23.966459 12063 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:23.966517 12063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:23.966706 12063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:23.968707 12063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "ad38b6b88a984d19a1a3216445a6073d"
format_stamp: "Formatted at 2026-09-28 06:40:23 on dist-test-slave-gtr8"
server_key: "9364bdf566255b94a5b947c47bdfa901"
server_key_iv: "eb2893eff2bd140b31c82737bcdd477f"
server_key_version: "encryptionkey@0"
I20260928 06:40:23.969210 12063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "ad38b6b88a984d19a1a3216445a6073d"
format_stamp: "Formatted at 2026-09-28 06:40:23 on dist-test-slave-gtr8"
server_key: "9364bdf566255b94a5b947c47bdfa901"
server_key_iv: "eb2893eff2bd140b31c82737bcdd477f"
server_key_version: "encryptionkey@0"
I20260928 06:40:23.972678 12063 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:23.974960 12078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:23.976191 12063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:23.976296 12063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "ad38b6b88a984d19a1a3216445a6073d"
format_stamp: "Formatted at 2026-09-28 06:40:23 on dist-test-slave-gtr8"
server_key: "9364bdf566255b94a5b947c47bdfa901"
server_key_iv: "eb2893eff2bd140b31c82737bcdd477f"
server_key_version: "encryptionkey@0"
I20260928 06:40:23.976390 12063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:23.990192 12063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:23.993160 12063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:23.993321 12063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:23.993875 12063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:23.994679 12063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:23.994724 12063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:23.994766 12063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:23.994793 12063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:24.003376 12063 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:45275
I20260928 06:40:24.003415 12191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:45275 every 8 connection(s)
I20260928 06:40:24.004273 12063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
Sep 28 06:40:24 dist-test-slave-gtr8 krb5kdc[11822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:24.014622 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 12063
I20260928 06:40:24.014739 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:24.015017 8799 external_mini_cluster.cc:1428] Setting key b94e97df4c0f71be8f936dee51f5832b
I20260928 06:40:24.017326 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:24.005979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:39065 (local address 127.8.151.254:44145)
0928 06:40:24.006130 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:24.006134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:24.006945 (+ 811us) server_negotiation.cc:407] Connection header received
0928 06:40:24.008306 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:24.008310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:24.008366 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:24.008439 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:24.009577 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:24.010103 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:24.011225 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:24.011456 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:24.011551 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:24.013749 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:24.013752 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:24.013754 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:24.013777 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:24.015328 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:24.015825 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:24.015828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:24.015829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:24.015933 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:24.016283 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:24.016287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:24.016288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:24.016463 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:24.016545 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:24.017037 (+ 492us) server_negotiation.cc:299] Negotiation successful
0928 06:40:24.017162 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 06:40:24.018321 12194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:24.006363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:44145 (local address 127.8.151.194:39065)
0928 06:40:24.006794 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:24.006836 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:24.008097 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:24.008623 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:24.008629 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:24.008976 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:24.009433 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:24.009446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:24.010216 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:24.010219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:24.011085 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:24.011094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:24.011591 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:24.011595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:24.011756 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:24.012252 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:24.012266 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:24.013591 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:24.015441 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:24.015447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:24.015463 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:24.015736 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:24.016018 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:24.016021 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:24.016022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:24.016195 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:24.016564 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:24.016570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:24.016806 (+ 236us) client_negotiation.cc:769] Sending connection context
0928 06:40:24.016970 (+ 164us) client_negotiation.cc:240] Negotiation successful
0928 06:40:24.017294 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260928 06:40:24.019541 12192 heartbeater.cc:347] Connected to a master server at 127.8.151.254:44145
I20260928 06:40:24.019783 12192 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:24.020270 12192 heartbeater.cc:511] Master 127.8.151.254:44145 requested a full tablet report, sending...
I20260928 06:40:24.021307 11867 ts_manager.cc:194] Registered new tserver with Master: ad38b6b88a984d19a1a3216445a6073d (127.8.151.194:45275)
I20260928 06:40:24.021922 11867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:39065
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:40:24.042215 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:44145
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:24.147429 12197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:24.147673 12197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:24.147730 12197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:24.151629 12197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:24.151687 12197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:24.151764 12197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:24.156867 12197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:44145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:24.157866 12197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:24.158845 12197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:24.168056 12205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:24.168020 12202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:24.168125 12197 server_base.cc:1034] running on GCE node
W20260928 06:40:24.168038 12203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:24.168618 12197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:24.169248 12197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:24.170399 12197 hybrid_clock.cc:648] HybridClock initialized: now 1790577624170378 us; error 33 us; skew 500 ppm
Sep 28 06:40:24 dist-test-slave-gtr8 krb5kdc[11822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:24.172984 12197 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:40:24.173834 12197 webserver.cc:460] Webserver started at http://127.8.151.195:46139/ using document root <none> and password file <none>
I20260928 06:40:24.174367 12197 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:24.174432 12197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:24.174656 12197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:24.176662 12197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "3953bfc9e9be494f8db0af818ce0bb26"
format_stamp: "Formatted at 2026-09-28 06:40:24 on dist-test-slave-gtr8"
server_key: "3e53d944fa88c571bee11deb910a523a"
server_key_iv: "9ff112ce3b8d5593eb4e0e04e37b1d83"
server_key_version: "encryptionkey@0"
I20260928 06:40:24.177187 12197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "3953bfc9e9be494f8db0af818ce0bb26"
format_stamp: "Formatted at 2026-09-28 06:40:24 on dist-test-slave-gtr8"
server_key: "3e53d944fa88c571bee11deb910a523a"
server_key_iv: "9ff112ce3b8d5593eb4e0e04e37b1d83"
server_key_version: "encryptionkey@0"
I20260928 06:40:24.180738 12197 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260928 06:40:24.183085 12212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:24.184227 12197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:40:24.184330 12197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "3953bfc9e9be494f8db0af818ce0bb26"
format_stamp: "Formatted at 2026-09-28 06:40:24 on dist-test-slave-gtr8"
server_key: "3e53d944fa88c571bee11deb910a523a"
server_key_iv: "9ff112ce3b8d5593eb4e0e04e37b1d83"
server_key_version: "encryptionkey@0"
I20260928 06:40:24.184422 12197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:24.200158 12197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:24.202994 12197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:24.203194 12197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:24.203820 12197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:24.204602 12197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:24.204649 12197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:24.204701 12197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:24.204730 12197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:24.213117 12197 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:41519
I20260928 06:40:24.213186 12325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:41519 every 8 connection(s)
I20260928 06:40:24.213997 12197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:24.217957 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 12197
I20260928 06:40:24.218087 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:24.218410 8799 external_mini_cluster.cc:1428] Setting key 1479f36ed0a2ef5b94cb37c1bb207810
Sep 28 06:40:24 dist-test-slave-gtr8 krb5kdc[11822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577624, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:24.226907 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:24.215802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:55611 (local address 127.8.151.254:44145)
0928 06:40:24.215943 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:24.215947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:24.216764 (+ 817us) server_negotiation.cc:407] Connection header received
0928 06:40:24.218256 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:24.218259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:24.218308 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:24.218381 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:24.219832 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:24.220382 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:24.221483 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:24.221706 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:24.221777 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:24.224006 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:24.224010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:24.224012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:24.224036 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:24.225140 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:24.225595 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:24.225598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:24.225599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:24.225667 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:24.225975 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:24.225977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:24.225978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:24.226167 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:24.226239 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:24.226688 (+ 449us) server_negotiation.cc:299] Negotiation successful
0928 06:40:24.226784 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:40:24.227782 12328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:24.216182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:44145 (local address 127.8.151.195:55611)
0928 06:40:24.216620 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:24.216663 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:24.218007 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:24.218587 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:24.218596 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:24.219022 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:24.219711 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:24.219731 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:24.220502 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:24.220506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:24.221390 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:24.221399 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:24.221821 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:24.221823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:24.221958 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:24.222457 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:24.222473 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:24.223865 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:24.225249 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:24.225254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:24.225265 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:24.225514 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:24.225766 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:24.225768 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:24.225770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:24.225905 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:24.226261 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:24.226266 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:24.226492 (+ 226us) client_negotiation.cc:769] Sending connection context
0928 06:40:24.226657 (+ 165us) client_negotiation.cc:240] Negotiation successful
0928 06:40:24.226899 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":105,"threads_started":1}
I20260928 06:40:24.228855 12326 heartbeater.cc:347] Connected to a master server at 127.8.151.254:44145
I20260928 06:40:24.229101 12326 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:24.229559 12326 heartbeater.cc:511] Master 127.8.151.254:44145 requested a full tablet report, sending...
I20260928 06:40:24.230466 11867 ts_manager.cc:194] Registered new tserver with Master: 3953bfc9e9be494f8db0af818ce0bb26 (127.8.151.195:41519)
I20260928 06:40:24.231072 11867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:55611
I20260928 06:40:24.232120 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 06:40:24 dist-test-slave-gtr8 krb5kdc[11822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577624, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 28 06:40:24 dist-test-slave-gtr8 krb5kdc[11822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577624, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:24.255820 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:24.248041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.8.151.254:44145)
0928 06:40:24.248158 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:24.248162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:24.248334 (+ 172us) server_negotiation.cc:407] Connection header received
0928 06:40:24.248428 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:24.248430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:24.248476 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:24.248550 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:24.249212 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:24.249734 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:24.250542 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:24.250757 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:24.250831 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:24.253041 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:24.253045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:24.253047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:24.253093 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:24.254376 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:24.254762 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:24.254765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:24.254766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:24.254840 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:24.255078 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:24.255081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:24.255082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:24.255330 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:24.255434 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:24.255601 (+ 167us) server_negotiation.cc:299] Negotiation successful
0928 06:40:24.255697 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260928 06:40:24.256206 11867 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43180
I20260928 06:40:24.257392 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 11929
I20260928 06:40:24.261842 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 12063
I20260928 06:40:24.265967 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 12197
I20260928 06:40:24.270125 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 11835
2026-09-28T06:40:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2004 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:24 dist-test-slave-gtr8 krb5kdc[12338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:24 dist-test-slave-gtr8 krb5kdc[12338](info): set up 2 sockets
Sep 28 06:40:24 dist-test-slave-gtr8 krb5kdc[12338](info): commencing operation
krb5kdc: starting...
W20260928 06:40:25.223454 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:25 dist-test-slave-gtr8 krb5kdc[12338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:25.338241 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:43649
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:43649
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:25.443584 12351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:25.443833 12351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:25.443874 12351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:25.447783 12351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:25.447837 12351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:25.447855 12351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:25.447871 12351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:25.447899 12351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:25.453141 12351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:43649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:43649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:25.454190 12351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:25.455240 12351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:25.463519 12356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:25.463500 12359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:25.463492 12357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:25.463606 12351 server_base.cc:1034] running on GCE node
I20260928 06:40:25.464282 12351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:25.465355 12351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:25.466512 12351 hybrid_clock.cc:648] HybridClock initialized: now 1790577625466493 us; error 31 us; skew 500 ppm
Sep 28 06:40:25 dist-test-slave-gtr8 krb5kdc[12338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577625, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:25.473678 12351 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:25.474562 12351 webserver.cc:460] Webserver started at http://127.8.151.254:39751/ using document root <none> and password file <none>
I20260928 06:40:25.475106 12351 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:25.475205 12351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:25.475406 12351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:25.477367 12351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "38071739eaed4df0a001180863932278"
format_stamp: "Formatted at 2026-09-28 06:40:25 on dist-test-slave-gtr8"
server_key: "7e69fe1fe65d768b304277947e73b7dd"
server_key_iv: "21cdb947e54ff38124b5263851b1637e"
server_key_version: "encryptionkey@0"
I20260928 06:40:25.477887 12351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "38071739eaed4df0a001180863932278"
format_stamp: "Formatted at 2026-09-28 06:40:25 on dist-test-slave-gtr8"
server_key: "7e69fe1fe65d768b304277947e73b7dd"
server_key_iv: "21cdb947e54ff38124b5263851b1637e"
server_key_version: "encryptionkey@0"
I20260928 06:40:25.481381 12351 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:25.483650 12366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:25.484759 12351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:25.484870 12351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "38071739eaed4df0a001180863932278"
format_stamp: "Formatted at 2026-09-28 06:40:25 on dist-test-slave-gtr8"
server_key: "7e69fe1fe65d768b304277947e73b7dd"
server_key_iv: "21cdb947e54ff38124b5263851b1637e"
server_key_version: "encryptionkey@0"
I20260928 06:40:25.484957 12351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:25.500952 12351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:25.503669 12351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:25.503837 12351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:25.510317 12351 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:43649
I20260928 06:40:25.510380 12418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:43649 every 8 connection(s)
I20260928 06:40:25.511250 12351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:25.513752 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 12351
I20260928 06:40:25.513823 12419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:25.513868 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:25.514145 8799 external_mini_cluster.cc:1428] Setting key 5443d435cc775ca11a685dbe54599df7
I20260928 06:40:25.520828 12419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278: Bootstrap starting.
I20260928 06:40:25.523278 12419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278: Neither blocks nor log segments found. Creating new log.
Sep 28 06:40:25 dist-test-slave-gtr8 krb5kdc[12338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:25.524071 12419 log.cc:824] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:25.525988 12419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278: No bootstrap required, opened a new log
I20260928 06:40:25.528112 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:25.515490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35876 (local address 127.8.151.254:43649)
0928 06:40:25.515892 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:25.515910 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:25.515951 (+ 41us) server_negotiation.cc:407] Connection header received
0928 06:40:25.516961 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:25.516985 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:25.517394 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:25.517745 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:25.518377 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:25.519878 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:25.520700 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:25.521350 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:25.521495 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:25.523860 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:25.523867 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:25.523882 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:25.523912 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:25.525460 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:25.525903 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:25.525907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:25.525912 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:25.526011 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:25.526325 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:25.526327 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:25.526328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:25.526682 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:25.526816 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:25.527134 (+ 318us) server_negotiation.cc:299] Negotiation successful
0928 06:40:25.527428 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":116,"threads_started":1}
I20260928 06:40:25.529649 12419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38071739eaed4df0a001180863932278" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43649 } }
I20260928 06:40:25.529860 12419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:25.529911 12419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38071739eaed4df0a001180863932278, State: Initialized, Role: FOLLOWER
I20260928 06:40:25.530328 12419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38071739eaed4df0a001180863932278" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43649 } }
I20260928 06:40:25.530464 12419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:25.530525 12419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:25.530627 12419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:25.531499 12419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38071739eaed4df0a001180863932278" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43649 } }
I20260928 06:40:25.531808 12419 leader_election.cc:302] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38071739eaed4df0a001180863932278; no voters:
I20260928 06:40:25.532075 12419 leader_election.cc:288] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:25.532168 12424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:25.532368 12424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [term 1 LEADER]: Becoming Leader. State: Replica: 38071739eaed4df0a001180863932278, State: Running, Role: LEADER
I20260928 06:40:25.532696 12424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38071739eaed4df0a001180863932278" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43649 } }
I20260928 06:40:25.533126 12419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:25.534102 12426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38071739eaed4df0a001180863932278. Latest consensus state: current_term: 1 leader_uuid: "38071739eaed4df0a001180863932278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38071739eaed4df0a001180863932278" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43649 } } }
I20260928 06:40:25.534216 12426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:25.534344 12425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38071739eaed4df0a001180863932278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38071739eaed4df0a001180863932278" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43649 } } }
I20260928 06:40:25.534487 12425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:25.534765 12433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:25.537355 12433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:25.542448 12433 catalog_manager.cc:1347] Generated new cluster ID: 2e7d1d067f884279911949eb0857e7ea
I20260928 06:40:25.542503 12433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:25.587845 12433 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:25.589203 12433 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:25.598971 12433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38071739eaed4df0a001180863932278: Generated new TSK 0
I20260928 06:40:25.599709 12433 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:40:25.630712 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:43649
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:25.744151 12445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:25.744390 12445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:25.744427 12445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:25.748405 12445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:25.748461 12445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:25.748522 12445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:25.753563 12445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:25.754496 12445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:25.755522 12445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:25.764827 12450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:25.764822 12451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:25.764952 12445 server_base.cc:1034] running on GCE node
W20260928 06:40:25.764904 12453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:25.765388 12445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:25.765945 12445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:25.767103 12445 hybrid_clock.cc:648] HybridClock initialized: now 1790577625767071 us; error 41 us; skew 500 ppm
Sep 28 06:40:25 dist-test-slave-gtr8 krb5kdc[12338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577625, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:25.769673 12445 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:40:25.770495 12445 webserver.cc:460] Webserver started at http://127.8.151.193:38299/ using document root <none> and password file <none>
I20260928 06:40:25.771006 12445 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:25.771068 12445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:25.771274 12445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:25.773217 12445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "2ae7af5f0ff34abf807adebabb309cca"
format_stamp: "Formatted at 2026-09-28 06:40:25 on dist-test-slave-gtr8"
server_key: "38ee5434bbf02babe6c74b44e24c609e"
server_key_iv: "1bf1cd0332488ae1bf27d53d5029b992"
server_key_version: "encryptionkey@0"
I20260928 06:40:25.773729 12445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "2ae7af5f0ff34abf807adebabb309cca"
format_stamp: "Formatted at 2026-09-28 06:40:25 on dist-test-slave-gtr8"
server_key: "38ee5434bbf02babe6c74b44e24c609e"
server_key_iv: "1bf1cd0332488ae1bf27d53d5029b992"
server_key_version: "encryptionkey@0"
I20260928 06:40:25.777447 12445 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:25.780150 12460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:25.781436 12445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:25.781541 12445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "2ae7af5f0ff34abf807adebabb309cca"
format_stamp: "Formatted at 2026-09-28 06:40:25 on dist-test-slave-gtr8"
server_key: "38ee5434bbf02babe6c74b44e24c609e"
server_key_iv: "1bf1cd0332488ae1bf27d53d5029b992"
server_key_version: "encryptionkey@0"
I20260928 06:40:25.781641 12445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:25.800175 12445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:25.802826 12445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:25.802994 12445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:25.803578 12445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:25.804340 12445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:25.804384 12445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:25.804435 12445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:25.804462 12445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:25.812552 12445 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:43925
I20260928 06:40:25.812588 12573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:43925 every 8 connection(s)
I20260928 06:40:25.813388 12445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:25.817034 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 12445
I20260928 06:40:25.817131 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:25.817391 8799 external_mini_cluster.cc:1428] Setting key 12c47e1e91da0181cced616ec8664ab4
Sep 28 06:40:25 dist-test-slave-gtr8 krb5kdc[12338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577625, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:40:25.826437 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:25.815192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:57409 (local address 127.8.151.254:43649)
0928 06:40:25.815325 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:25.815329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:25.816137 (+ 808us) server_negotiation.cc:407] Connection header received
0928 06:40:25.817531 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:25.817535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:25.817591 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:25.817667 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:25.818887 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:25.819450 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:25.820558 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:25.820779 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:25.820862 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:25.823175 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:25.823179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:25.823182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:25.823206 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:25.824526 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:25.824983 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:25.824986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:25.824988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:25.825089 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:25.825386 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:25.825390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:25.825391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:25.825561 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:25.825643 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:25.826153 (+ 510us) server_negotiation.cc:299] Negotiation successful
0928 06:40:25.826273 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:40:25.827276 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:25.815546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:43649 (local address 127.8.151.193:57409)
0928 06:40:25.815970 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:25.816011 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:25.817325 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:25.817841 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:25.817849 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:25.818233 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:25.818755 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:25.818768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:25.819576 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:25.819579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:25.820441 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:25.820450 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:25.820910 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:25.820914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:25.821069 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:25.821583 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:25.821599 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:25.822996 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:25.824636 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:25.824642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:25.824656 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:25.824894 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:25.825171 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:25.825174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:25.825176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:25.825304 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:25.825654 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:25.825660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:25.825913 (+ 253us) client_negotiation.cc:769] Sending connection context
0928 06:40:25.826087 (+ 174us) client_negotiation.cc:240] Negotiation successful
0928 06:40:25.826407 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":102,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:40:25.828434 12574 heartbeater.cc:347] Connected to a master server at 127.8.151.254:43649
I20260928 06:40:25.828718 12574 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:25.829174 12574 heartbeater.cc:511] Master 127.8.151.254:43649 requested a full tablet report, sending...
I20260928 06:40:25.830806 12383 ts_manager.cc:194] Registered new tserver with Master: 2ae7af5f0ff34abf807adebabb309cca (127.8.151.193:43925)
I20260928 06:40:25.831997 12383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:57409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:40:25.846442 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:43649
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:25.951869 12579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:25.952131 12579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:25.952188 12579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:25.956120 12579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:25.956174 12579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:25.956252 12579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:25.961455 12579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:25.962433 12579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:25.963434 12579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:25.972636 12585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:25.972741 12579 server_base.cc:1034] running on GCE node
W20260928 06:40:25.972600 12584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:25.972607 12587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:25.973224 12579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:25.973803 12579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:25.974977 12579 hybrid_clock.cc:648] HybridClock initialized: now 1790577625974964 us; error 59 us; skew 500 ppm
Sep 28 06:40:25 dist-test-slave-gtr8 krb5kdc[12338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577625, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:25.977651 12579 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:40:25.978556 12579 webserver.cc:460] Webserver started at http://127.8.151.194:38505/ using document root <none> and password file <none>
I20260928 06:40:25.979087 12579 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:25.979194 12579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:25.979403 12579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:25.981418 12579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "65e0b81a29fc4812832d49d04f0aa5b9"
format_stamp: "Formatted at 2026-09-28 06:40:25 on dist-test-slave-gtr8"
server_key: "b66927228f81c883a9b1c7857a974f67"
server_key_iv: "22698fa668ca48a81aeb0528d0987bef"
server_key_version: "encryptionkey@0"
I20260928 06:40:25.981950 12579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "65e0b81a29fc4812832d49d04f0aa5b9"
format_stamp: "Formatted at 2026-09-28 06:40:25 on dist-test-slave-gtr8"
server_key: "b66927228f81c883a9b1c7857a974f67"
server_key_iv: "22698fa668ca48a81aeb0528d0987bef"
server_key_version: "encryptionkey@0"
I20260928 06:40:25.985635 12579 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260928 06:40:25.987984 12594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:25.989255 12579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:25.989367 12579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "65e0b81a29fc4812832d49d04f0aa5b9"
format_stamp: "Formatted at 2026-09-28 06:40:25 on dist-test-slave-gtr8"
server_key: "b66927228f81c883a9b1c7857a974f67"
server_key_iv: "22698fa668ca48a81aeb0528d0987bef"
server_key_version: "encryptionkey@0"
I20260928 06:40:25.989459 12579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:26.023782 12579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:26.026823 12579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:26.026995 12579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:26.027611 12579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:26.028403 12579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:26.028457 12579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:26.028512 12579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:26.028537 12579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:26.037304 12579 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:39205
I20260928 06:40:26.037335 12707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:39205 every 8 connection(s)
I20260928 06:40:26.038156 12579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:26.042794 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 12579
I20260928 06:40:26.042892 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:26.043161 8799 external_mini_cluster.cc:1428] Setting key 9c430d08a5abe2a9839bedaf50bd654d
Sep 28 06:40:26 dist-test-slave-gtr8 krb5kdc[12338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577625, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:26.051487 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:26.039908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:37077 (local address 127.8.151.254:43649)
0928 06:40:26.040033 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:26.040037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:26.040855 (+ 818us) server_negotiation.cc:407] Connection header received
0928 06:40:26.042187 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:26.042190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:26.042254 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:26.042326 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:26.043517 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.044129 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.045224 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.045436 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.045527 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:26.047900 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:26.047904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:26.047907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:26.047935 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:26.049339 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:26.049927 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:26.049929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:26.049931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:26.050009 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:26.050375 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:26.050377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:26.050379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:26.050544 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:26.050618 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:26.051259 (+ 641us) server_negotiation.cc:299] Negotiation successful
0928 06:40:26.051364 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
I20260928 06:40:26.052584 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:26.040268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:43649 (local address 127.8.151.194:37077)
0928 06:40:26.040703 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:26.040743 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:26.041993 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:26.042501 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:26.042508 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:26.042835 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:26.043383 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:26.043398 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.044247 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.044250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:26.045117 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:26.045126 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.045573 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.045577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:26.045726 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:26.046272 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:26.046289 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:26.047719 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:26.049452 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:26.049460 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:26.049475 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:26.049820 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:26.050112 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:26.050115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:26.050117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:26.050296 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:26.050639 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:26.050646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:26.050968 (+ 322us) client_negotiation.cc:769] Sending connection context
0928 06:40:26.051215 (+ 247us) client_negotiation.cc:240] Negotiation successful
0928 06:40:26.051583 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":95,"threads_started":1}
I20260928 06:40:26.053671 12708 heartbeater.cc:347] Connected to a master server at 127.8.151.254:43649
I20260928 06:40:26.053890 12708 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:26.054412 12708 heartbeater.cc:511] Master 127.8.151.254:43649 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
I20260928 06:40:26.055545 12383 ts_manager.cc:194] Registered new tserver with Master: 65e0b81a29fc4812832d49d04f0aa5b9 (127.8.151.194:39205)
I20260928 06:40:26.056243 12383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:37077
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:40:26.069685 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:43649
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:26.176844 12713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:26.177094 12713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:26.177153 12713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:26.181147 12713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:26.181203 12713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:26.181278 12713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:26.186439 12713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:26.187472 12713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:26.188470 12713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:26.197743 12718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:26.197813 12721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:26.197894 12713 server_base.cc:1034] running on GCE node
W20260928 06:40:26.197829 12719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:26.198431 12713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:26.199065 12713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:26.200215 12713 hybrid_clock.cc:648] HybridClock initialized: now 1790577626200188 us; error 41 us; skew 500 ppm
Sep 28 06:40:26 dist-test-slave-gtr8 krb5kdc[12338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577626, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:26.202818 12713 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:40:26.203802 12713 webserver.cc:460] Webserver started at http://127.8.151.195:42609/ using document root <none> and password file <none>
I20260928 06:40:26.204377 12713 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:26.204460 12713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:26.204694 12713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:26.206774 12713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "c2376bb6955e4d0ab8862baca04e1a34"
format_stamp: "Formatted at 2026-09-28 06:40:26 on dist-test-slave-gtr8"
server_key: "65e6e7ecb60b83e194ea61f5914fe82d"
server_key_iv: "b87fb40c2d81ec7b0136068aaa39e15c"
server_key_version: "encryptionkey@0"
I20260928 06:40:26.207351 12713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "c2376bb6955e4d0ab8862baca04e1a34"
format_stamp: "Formatted at 2026-09-28 06:40:26 on dist-test-slave-gtr8"
server_key: "65e6e7ecb60b83e194ea61f5914fe82d"
server_key_iv: "b87fb40c2d81ec7b0136068aaa39e15c"
server_key_version: "encryptionkey@0"
I20260928 06:40:26.211083 12713 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:26.213459 12728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:26.214653 12713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:26.214777 12713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "c2376bb6955e4d0ab8862baca04e1a34"
format_stamp: "Formatted at 2026-09-28 06:40:26 on dist-test-slave-gtr8"
server_key: "65e6e7ecb60b83e194ea61f5914fe82d"
server_key_iv: "b87fb40c2d81ec7b0136068aaa39e15c"
server_key_version: "encryptionkey@0"
I20260928 06:40:26.214872 12713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:26.249758 12713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:26.252919 12713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:26.253101 12713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:26.253710 12713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:26.254562 12713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:26.254611 12713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:26.254673 12713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:26.254712 12713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:26.263965 12713 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:40691
I20260928 06:40:26.263991 12841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:40691 every 8 connection(s)
I20260928 06:40:26.264858 12713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:26.265748 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 12713
I20260928 06:40:26.265868 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:26.266167 8799 external_mini_cluster.cc:1428] Setting key 4fcccdc69c21a9cbbec04bdfbb65c207
Sep 28 06:40:26 dist-test-slave-gtr8 krb5kdc[12338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577626, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:26.277750 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:26.266766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:48645 (local address 127.8.151.254:43649)
0928 06:40:26.266910 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:26.266915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:26.267922 (+ 1007us) server_negotiation.cc:407] Connection header received
0928 06:40:26.269364 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:26.269367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:26.269411 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:26.269467 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:26.270621 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.271216 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.272356 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.272579 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.272665 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:26.274960 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:26.274963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:26.274965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:26.274991 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:26.276101 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:26.276527 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:26.276530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:26.276531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:26.276597 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:26.276854 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:26.276856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:26.276857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:26.277014 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:26.277081 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:26.277519 (+ 438us) server_negotiation.cc:299] Negotiation successful
0928 06:40:26.277627 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 06:40:26.278548 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:26.267257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:43649 (local address 127.8.151.195:48645)
0928 06:40:26.267763 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:26.267818 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:26.269160 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:26.269613 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:26.269620 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:26.269984 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:26.270469 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:26.270482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.271323 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.271326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:26.272234 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:26.272244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.272700 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.272702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:26.272838 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:26.273348 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:26.273364 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:26.274809 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:26.276201 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:26.276206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:26.276217 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:26.276453 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:26.276669 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:26.276671 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:26.276672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:26.276793 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:26.277094 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:26.277099 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:26.277319 (+ 220us) client_negotiation.cc:769] Sending connection context
0928 06:40:26.277467 (+ 148us) client_negotiation.cc:240] Negotiation successful
0928 06:40:26.277715 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":124,"threads_started":1}
I20260928 06:40:26.279815 12842 heartbeater.cc:347] Connected to a master server at 127.8.151.254:43649
I20260928 06:40:26.280037 12842 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:26.280490 12842 heartbeater.cc:511] Master 127.8.151.254:43649 requested a full tablet report, sending...
I20260928 06:40:26.281469 12383 ts_manager.cc:194] Registered new tserver with Master: c2376bb6955e4d0ab8862baca04e1a34 (127.8.151.195:40691)
I20260928 06:40:26.282003 12383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:48645
I20260928 06:40:26.291378 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 06:40:26 dist-test-slave-gtr8 krb5kdc[12338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 06:40:26 dist-test-slave-gtr8 krb5kdc[12338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:40:26.316385 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:26.306938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54330 (local address 127.8.151.193:43925)
0928 06:40:26.307268 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:26.307284 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:26.307311 (+ 27us) server_negotiation.cc:407] Connection header received
0928 06:40:26.307360 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:26.307365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:26.307548 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:26.307667 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:26.308431 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.309228 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.310134 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.310535 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.310629 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:26.313006 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:26.313015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:26.313028 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:26.313053 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:26.314626 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:26.315037 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:26.315042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:26.315043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:26.315173 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:26.315418 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:26.315421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:26.315423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:26.315665 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:26.315855 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:26.316058 (+ 203us) server_negotiation.cc:299] Negotiation successful
0928 06:40:26.316221 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":107,"threads_started":1}
Sep 28 06:40:26 dist-test-slave-gtr8 krb5kdc[12338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:26.326819 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:26.319397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.8.151.254:43649)
0928 06:40:26.319531 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:26.319535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:26.319567 (+ 32us) server_negotiation.cc:407] Connection header received
0928 06:40:26.319690 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:26.319692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:26.319733 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:26.319795 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:26.320341 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.320877 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.321580 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:26.321807 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:26.321885 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:26.323980 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:26.323983 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:26.323985 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:26.324009 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:26.325208 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:26.325582 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:26.325585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:26.325586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:26.325683 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:26.325943 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:26.325945 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:26.325946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:26.326165 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:26.326277 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:26.326544 (+ 267us) server_negotiation.cc:299] Negotiation successful
0928 06:40:26.326661 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260928 06:40:26.327294 12383 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35886
I20260928 06:40:26.327888 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 12445
I20260928 06:40:26.332849 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 12579
I20260928 06:40:26.337332 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 12713
I20260928 06:40:26.341472 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 12351
2026-09-28T06:40:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2070 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:26 dist-test-slave-gtr8 krb5kdc[12854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:26 dist-test-slave-gtr8 krb5kdc[12854](info): set up 2 sockets
Sep 28 06:40:26 dist-test-slave-gtr8 krb5kdc[12854](info): commencing operation
krb5kdc: starting...
W20260928 06:40:27.287544 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:27 dist-test-slave-gtr8 krb5kdc[12854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:27.405665 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:35793
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:37997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:35793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:27.513804 12867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:27.514078 12867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:27.514159 12867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:27.518149 12867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:27.518203 12867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:27.518226 12867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:27.518249 12867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:27.518277 12867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:27.523393 12867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:37997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:35793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:35793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:27.524366 12867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:27.525334 12867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:27.533457 12873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:27.533492 12872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:27.533481 12875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:27.533473 12867 server_base.cc:1034] running on GCE node
I20260928 06:40:27.534224 12867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:27.535348 12867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:27.536518 12867 hybrid_clock.cc:648] HybridClock initialized: now 1790577627536485 us; error 61 us; skew 500 ppm
Sep 28 06:40:27 dist-test-slave-gtr8 krb5kdc[12854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:27.539283 12867 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:27.539937 12867 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:27.539994 12867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:27.540159 12867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:27.542097 12867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "bcc4c0548ceb44b9ac61e09ef5552293"
format_stamp: "Formatted at 2026-09-28 06:40:27 on dist-test-slave-gtr8"
server_key: "f0a651041d387b7e586e624fe9271bf7"
server_key_iv: "3f4f0bf1c01e1686dda62efd80d7d156"
server_key_version: "encryptionkey@0"
I20260928 06:40:27.542577 12867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "bcc4c0548ceb44b9ac61e09ef5552293"
format_stamp: "Formatted at 2026-09-28 06:40:27 on dist-test-slave-gtr8"
server_key: "f0a651041d387b7e586e624fe9271bf7"
server_key_iv: "3f4f0bf1c01e1686dda62efd80d7d156"
server_key_version: "encryptionkey@0"
I20260928 06:40:27.546389 12867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:40:27.548807 12881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:27.550019 12867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:27.550176 12867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "bcc4c0548ceb44b9ac61e09ef5552293"
format_stamp: "Formatted at 2026-09-28 06:40:27 on dist-test-slave-gtr8"
server_key: "f0a651041d387b7e586e624fe9271bf7"
server_key_iv: "3f4f0bf1c01e1686dda62efd80d7d156"
server_key_version: "encryptionkey@0"
I20260928 06:40:27.550275 12867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:27.569063 12867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:27.571853 12867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:27.572021 12867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:27.578382 12867 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:35793
I20260928 06:40:27.578626 12933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:35793 every 8 connection(s)
I20260928 06:40:27.578981 12867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:27.581423 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 12867
I20260928 06:40:27.581554 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:27.581821 12934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:27.581866 8799 external_mini_cluster.cc:1428] Setting key da8c7b2e3712515472444865c30d31dd
I20260928 06:40:27.588778 12934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293: Bootstrap starting.
Sep 28 06:40:27 dist-test-slave-gtr8 krb5kdc[12854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:27.591178 12934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:27.591996 12934 log.cc:824] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:27.593873 12934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293: No bootstrap required, opened a new log
I20260928 06:40:27.595860 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:27.583185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45060 (local address 127.8.151.254:35793)
0928 06:40:27.583633 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:27.583651 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:27.583703 (+ 52us) server_negotiation.cc:407] Connection header received
0928 06:40:27.584443 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:27.584465 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:27.584886 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:27.585254 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:27.585864 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:27.587314 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:27.588186 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:27.588861 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:27.589012 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:27.591517 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:27.591523 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:27.591536 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:27.591564 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:27.593191 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:27.593588 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:27.593593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:27.593597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:27.593707 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:27.593951 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:27.593954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:27.593956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:27.594361 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:27.594543 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:27.594820 (+ 277us) server_negotiation.cc:299] Negotiation successful
0928 06:40:27.595092 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":141,"threads_started":1}
I20260928 06:40:27.597677 12934 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc4c0548ceb44b9ac61e09ef5552293" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35793 } }
I20260928 06:40:27.597935 12934 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:27.597987 12934 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcc4c0548ceb44b9ac61e09ef5552293, State: Initialized, Role: FOLLOWER
I20260928 06:40:27.598392 12934 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc4c0548ceb44b9ac61e09ef5552293" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35793 } }
I20260928 06:40:27.598510 12934 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:27.598552 12934 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:27.598623 12934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:27.599536 12934 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc4c0548ceb44b9ac61e09ef5552293" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35793 } }
I20260928 06:40:27.599830 12934 leader_election.cc:302] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcc4c0548ceb44b9ac61e09ef5552293; no voters:
I20260928 06:40:27.600283 12934 leader_election.cc:288] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:27.600370 12939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:27.600590 12939 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [term 1 LEADER]: Becoming Leader. State: Replica: bcc4c0548ceb44b9ac61e09ef5552293, State: Running, Role: LEADER
I20260928 06:40:27.600960 12939 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc4c0548ceb44b9ac61e09ef5552293" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35793 } }
I20260928 06:40:27.601598 12934 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:27.602483 12940 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcc4c0548ceb44b9ac61e09ef5552293" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc4c0548ceb44b9ac61e09ef5552293" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35793 } } }
I20260928 06:40:27.602625 12940 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:27.603008 12948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:27.602989 12941 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcc4c0548ceb44b9ac61e09ef5552293. Latest consensus state: current_term: 1 leader_uuid: "bcc4c0548ceb44b9ac61e09ef5552293" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc4c0548ceb44b9ac61e09ef5552293" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35793 } } }
I20260928 06:40:27.603176 12941 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:27.605911 12948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:27.610833 12948 catalog_manager.cc:1347] Generated new cluster ID: 3922f85a36a4407c80b540b43aca6d49
I20260928 06:40:27.610898 12948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:27.617487 12948 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:27.618350 12948 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:27.623468 12948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293: Generated new TSK 0
I20260928 06:40:27.624058 12948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:40:27.653903 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35793
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:37997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:27.762704 12960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:27.762957 12960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:27.763018 12960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:27.767014 12960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:27.767074 12960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:27.767175 12960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:27.772342 12960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:37997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:27.773360 12960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:27.774374 12960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:27.783596 12968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:27.783633 12965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:27.783783 12960 server_base.cc:1034] running on GCE node
W20260928 06:40:27.783600 12966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:27.784264 12960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:27.784905 12960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:27.786075 12960 hybrid_clock.cc:648] HybridClock initialized: now 1790577627786053 us; error 32 us; skew 500 ppm
Sep 28 06:40:27 dist-test-slave-gtr8 krb5kdc[12854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:27.788863 12960 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:40:27.789628 12960 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:27.789700 12960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:27.789889 12960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:27.791980 12960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "236aba066969449ea3d2ba5e9ac29669"
format_stamp: "Formatted at 2026-09-28 06:40:27 on dist-test-slave-gtr8"
server_key: "01aea1dcb9b4af6c654d9f077d3a072c"
server_key_iv: "bc6292544e42e15ba5fbf7d9ad2678c1"
server_key_version: "encryptionkey@0"
I20260928 06:40:27.792498 12960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "236aba066969449ea3d2ba5e9ac29669"
format_stamp: "Formatted at 2026-09-28 06:40:27 on dist-test-slave-gtr8"
server_key: "01aea1dcb9b4af6c654d9f077d3a072c"
server_key_iv: "bc6292544e42e15ba5fbf7d9ad2678c1"
server_key_version: "encryptionkey@0"
I20260928 06:40:27.796197 12960 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:40:27.798576 12974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:27.799955 12960 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:40:27.800073 12960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "236aba066969449ea3d2ba5e9ac29669"
format_stamp: "Formatted at 2026-09-28 06:40:27 on dist-test-slave-gtr8"
server_key: "01aea1dcb9b4af6c654d9f077d3a072c"
server_key_iv: "bc6292544e42e15ba5fbf7d9ad2678c1"
server_key_version: "encryptionkey@0"
I20260928 06:40:27.800179 12960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:27.823912 12960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:27.827077 12960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:27.827273 12960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:27.827857 12960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:27.828676 12960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:27.828723 12960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:27.828776 12960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:27.828802 12960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:27.837054 12960 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:45071
I20260928 06:40:27.837076 13087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:45071 every 8 connection(s)
I20260928 06:40:27.837633 12960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:27.840518 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 12960
I20260928 06:40:27.840641 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:27.840955 8799 external_mini_cluster.cc:1428] Setting key 2b848bf6939e85464f67b52d57102d06
Sep 28 06:40:27 dist-test-slave-gtr8 krb5kdc[12854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:40:27.852344 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:27.839447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:38731 (local address 127.8.151.254:35793)
0928 06:40:27.839601 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:27.839606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:27.840511 (+ 905us) server_negotiation.cc:407] Connection header received
0928 06:40:27.842013 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:27.842016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:27.842065 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:27.842149 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:27.843666 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:27.844386 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:27.845666 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:27.845996 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:27.846112 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:27.848789 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:27.848793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:27.848796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:27.848828 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:27.850350 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:27.850855 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:27.850858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:27.850859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:27.850934 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:27.851345 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:27.851348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:27.851350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:27.851528 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:27.851599 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:27.852087 (+ 488us) server_negotiation.cc:299] Negotiation successful
0928 06:40:27.852219 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 06:40:27.853206 13090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:27.839864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:35793 (local address 127.8.151.193:38731)
0928 06:40:27.840345 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:27.840403 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:27.841813 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:27.842330 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:27.842340 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:27.842833 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:27.843510 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:27.843528 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:27.844541 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:27.844544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:27.845528 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:27.845537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:27.846165 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:27.846169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:27.846354 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:27.846919 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:27.846935 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:27.848615 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:27.850487 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:27.850494 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:27.850510 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:27.850767 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:27.851033 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:27.851036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:27.851038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:27.851234 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:27.851611 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:27.851619 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:27.851859 (+ 240us) client_negotiation.cc:769] Sending connection context
0928 06:40:27.852010 (+ 151us) client_negotiation.cc:240] Negotiation successful
0928 06:40:27.852301 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:40:27.854521 13088 heartbeater.cc:347] Connected to a master server at 127.8.151.254:35793
I20260928 06:40:27.854822 13088 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:27.855326 13088 heartbeater.cc:511] Master 127.8.151.254:35793 requested a full tablet report, sending...
I20260928 06:40:27.856812 12893 ts_manager.cc:194] Registered new tserver with Master: 236aba066969449ea3d2ba5e9ac29669 (127.8.151.193:45071)
I20260928 06:40:27.857937 12893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:38731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:40:27.870185 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35793
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:37997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:27.978655 13093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:27.978915 13093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:27.978962 13093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:27.982923 13093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:27.982980 13093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:27.983040 13093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:27.988289 13093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:37997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:27.989312 13093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:27.990293 13093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:27.999413 13099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:27.999413 13101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:27.999410 13098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:27.999464 13093 server_base.cc:1034] running on GCE node
I20260928 06:40:28.000036 13093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:28.000680 13093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:28.001848 13093 hybrid_clock.cc:648] HybridClock initialized: now 1790577628001826 us; error 33 us; skew 500 ppm
Sep 28 06:40:28 dist-test-slave-gtr8 krb5kdc[12854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577628, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:28.004663 13093 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:40:28.005412 13093 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:28.005479 13093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:28.005667 13093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:28.007692 13093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "4cee44e363214cdaaa83d79b218b7400"
format_stamp: "Formatted at 2026-09-28 06:40:28 on dist-test-slave-gtr8"
server_key: "0598031beb62283ce766f1592e58c05f"
server_key_iv: "3d119cb0125012d6343371e8c5eb7971"
server_key_version: "encryptionkey@0"
I20260928 06:40:28.008212 13093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "4cee44e363214cdaaa83d79b218b7400"
format_stamp: "Formatted at 2026-09-28 06:40:28 on dist-test-slave-gtr8"
server_key: "0598031beb62283ce766f1592e58c05f"
server_key_iv: "3d119cb0125012d6343371e8c5eb7971"
server_key_version: "encryptionkey@0"
I20260928 06:40:28.011755 13093 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:40:28.014081 13107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:28.015326 13093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260928 06:40:28.015429 13093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "4cee44e363214cdaaa83d79b218b7400"
format_stamp: "Formatted at 2026-09-28 06:40:28 on dist-test-slave-gtr8"
server_key: "0598031beb62283ce766f1592e58c05f"
server_key_iv: "3d119cb0125012d6343371e8c5eb7971"
server_key_version: "encryptionkey@0"
I20260928 06:40:28.015522 13093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:28.025414 13093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:28.028189 13093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:28.028353 13093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:28.028862 13093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:28.029618 13093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:28.029662 13093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:28.029711 13093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:28.029739 13093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:28.037997 13093 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:39039
I20260928 06:40:28.038018 13220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:39039 every 8 connection(s)
I20260928 06:40:28.038551 13093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:28.046155 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 13093
I20260928 06:40:28.046271 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:28.046604 8799 external_mini_cluster.cc:1428] Setting key 2fb22931c1480216cd4cdb730472ea75
Sep 28 06:40:28 dist-test-slave-gtr8 krb5kdc[12854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577628, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:28.051430 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.040307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:45081 (local address 127.8.151.254:35793)
0928 06:40:28.040433 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:28.040436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:28.041251 (+ 815us) server_negotiation.cc:407] Connection header received
0928 06:40:28.042475 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:28.042478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:28.042524 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:28.042600 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:28.043789 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.044324 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.045385 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.045592 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.045661 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.048276 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:28.048279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:28.048280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:28.048303 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:28.049529 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:28.049971 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:28.049974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:28.049975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:28.050093 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:28.050390 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:28.050393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:28.050395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:28.050571 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:28.050654 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:28.051126 (+ 472us) server_negotiation.cc:299] Negotiation successful
0928 06:40:28.051271 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 06:40:28.052271 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.040646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:35793 (local address 127.8.151.194:45081)
0928 06:40:28.041085 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:28.041129 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:28.042205 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:28.042777 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:28.042784 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:28.043179 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:28.043649 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.043664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.044419 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.044421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.045291 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.045300 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.045700 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.045703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.045833 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.046467 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:28.046488 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:28.048120 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:28.049632 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:28.049637 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:28.049652 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:28.049880 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:28.050179 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:28.050182 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:28.050184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:28.050308 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:28.050668 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:28.050675 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:28.050908 (+ 233us) client_negotiation.cc:769] Sending connection context
0928 06:40:28.051069 (+ 161us) client_negotiation.cc:240] Negotiation successful
0928 06:40:28.051409 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260928 06:40:28.053548 13221 heartbeater.cc:347] Connected to a master server at 127.8.151.254:35793
I20260928 06:40:28.053782 13221 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:28.054256 13221 heartbeater.cc:511] Master 127.8.151.254:35793 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
I20260928 06:40:28.055400 12893 ts_manager.cc:194] Registered new tserver with Master: 4cee44e363214cdaaa83d79b218b7400 (127.8.151.194:39039)
I20260928 06:40:28.056005 12893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:45081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:40:28.072573 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35793
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:37997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:28.183920 13226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:28.184197 13226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:28.184257 13226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:28.188254 13226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:28.188323 13226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:28.188405 13226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:28.193732 13226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:37997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:28.194832 13226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:28.195920 13226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:28.205727 13231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:28.205695 13234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:28.205863 13226 server_base.cc:1034] running on GCE node
W20260928 06:40:28.205739 13232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:28.206269 13226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:28.206892 13226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:28.208076 13226 hybrid_clock.cc:648] HybridClock initialized: now 1790577628208030 us; error 54 us; skew 500 ppm
Sep 28 06:40:28 dist-test-slave-gtr8 krb5kdc[12854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577628, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:28.210889 13226 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:40:28.211642 13226 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:28.211717 13226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:28.211930 13226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:28.213972 13226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "5bf1922a8af948cfb79c0a57e645582e"
format_stamp: "Formatted at 2026-09-28 06:40:28 on dist-test-slave-gtr8"
server_key: "67e04231847e18d22a3acfbb8709503e"
server_key_iv: "7f0c1a259952fe77ec41fc538f20214f"
server_key_version: "encryptionkey@0"
I20260928 06:40:28.214545 13226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "5bf1922a8af948cfb79c0a57e645582e"
format_stamp: "Formatted at 2026-09-28 06:40:28 on dist-test-slave-gtr8"
server_key: "67e04231847e18d22a3acfbb8709503e"
server_key_iv: "7f0c1a259952fe77ec41fc538f20214f"
server_key_version: "encryptionkey@0"
I20260928 06:40:28.218356 13226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:40:28.220878 13240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:28.222126 13226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260928 06:40:28.222242 13226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "5bf1922a8af948cfb79c0a57e645582e"
format_stamp: "Formatted at 2026-09-28 06:40:28 on dist-test-slave-gtr8"
server_key: "67e04231847e18d22a3acfbb8709503e"
server_key_iv: "7f0c1a259952fe77ec41fc538f20214f"
server_key_version: "encryptionkey@0"
I20260928 06:40:28.222342 13226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:28.235671 13226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:28.238368 13226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:28.238535 13226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:28.239084 13226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:28.239887 13226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:28.239935 13226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:28.239985 13226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:28.240015 13226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:28.248664 13226 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:41223
I20260928 06:40:28.248692 13353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:41223 every 8 connection(s)
I20260928 06:40:28.249221 13226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
Sep 28 06:40:28 dist-test-slave-gtr8 krb5kdc[12854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577628, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:28.259058 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 13226
I20260928 06:40:28.259196 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:28.259478 8799 external_mini_cluster.cc:1428] Setting key 4dca681bae5432f80010e591ad237a14
I20260928 06:40:28.262243 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.250991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:35791 (local address 127.8.151.254:35793)
0928 06:40:28.251182 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:28.251187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:28.252017 (+ 830us) server_negotiation.cc:407] Connection header received
0928 06:40:28.253323 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:28.253327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:28.253386 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:28.253456 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:28.254680 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.255257 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.256337 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.256546 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.256638 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.258868 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:28.258871 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:28.258873 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:28.258897 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:28.260267 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:28.260741 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:28.260744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:28.260745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:28.260840 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:28.261135 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:28.261138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:28.261140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:28.261354 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:28.261447 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:28.261989 (+ 542us) server_negotiation.cc:299] Negotiation successful
0928 06:40:28.262119 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 06:40:28.262934 13356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.251392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:35793 (local address 127.8.151.195:35791)
0928 06:40:28.251859 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:28.251907 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:28.253127 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:28.253642 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:28.253649 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:28.254060 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:28.254543 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.254556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.255369 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.255372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.256227 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.256236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.256688 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.256691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.256846 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.257349 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:28.257365 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:28.258720 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:28.260371 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:28.260377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:28.260391 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:28.260646 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:28.260923 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:28.260926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:28.260927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:28.261057 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:28.261473 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:28.261481 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:28.261741 (+ 260us) client_negotiation.cc:769] Sending connection context
0928 06:40:28.261903 (+ 162us) client_negotiation.cc:240] Negotiation successful
0928 06:40:28.262159 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":119,"threads_started":1}
I20260928 06:40:28.263998 13354 heartbeater.cc:347] Connected to a master server at 127.8.151.254:35793
I20260928 06:40:28.264237 13354 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:28.264680 13354 heartbeater.cc:511] Master 127.8.151.254:35793 requested a full tablet report, sending...
I20260928 06:40:28.265664 12893 ts_manager.cc:194] Registered new tserver with Master: 5bf1922a8af948cfb79c0a57e645582e (127.8.151.195:41223)
I20260928 06:40:28.266232 12893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:35791
I20260928 06:40:28.273361 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:40:28.282009 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.275242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.8.151.254:35793)
0928 06:40:28.275366 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:28.275372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:28.275567 (+ 195us) server_negotiation.cc:407] Connection header received
0928 06:40:28.275699 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:28.275702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:28.275746 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:28.275807 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:28.276458 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.276965 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.277741 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.277953 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.278036 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.279249 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:28.279253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:28.279256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:28.279281 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:28.280508 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:28.280887 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:28.280891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:28.280892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:28.280962 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:28.281196 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:28.281200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:28.281201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:28.281412 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:28.281532 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:28.281756 (+ 224us) server_negotiation.cc:299] Negotiation successful
0928 06:40:28.281864 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 06:40:28.284785 12893 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:28.287746 12893 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:28.301419 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.296018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:45071 (local address 127.0.0.1:38526)
0928 06:40:28.296381 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:28.296428 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:28.296575 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:28.297253 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:28.297259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:28.297284 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:28.297409 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.297415 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.298683 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.298686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.300125 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.300133 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.300875 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.300879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.300983 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.301017 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 06:40:28.301141 (+ 124us) client_negotiation.cc:240] Negotiation successful
0928 06:40:28.301265 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":47,"thread_start_us":76,"threads_started":1}
I20260928 06:40:28.301862 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.296188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.8.151.193:45071)
0928 06:40:28.296493 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:28.296505 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:28.296531 (+ 26us) server_negotiation.cc:407] Connection header received
0928 06:40:28.296754 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:28.296761 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:28.296977 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:28.297162 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:28.297521 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.298530 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.300229 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.300768 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.300862 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.301010 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:28.301621 (+ 611us) server_negotiation.cc:299] Negotiation successful
0928 06:40:28.301721 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":108,"threads_started":1}
I20260928 06:40:28.302474 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.296783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:41223 (local address 127.0.0.1:48290)
0928 06:40:28.297336 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:28.297349 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:28.297430 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:28.298217 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:28.298220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:28.298238 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:28.298338 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.298343 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.299751 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.299755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.301071 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.301077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.302097 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.302101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.302203 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.302226 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:40:28.302293 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 06:40:28.302350 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":97,"thread_start_us":85,"threads_started":1}
I20260928 06:40:28.303493 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.297073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:39039 (local address 127.0.0.1:46542)
0928 06:40:28.297627 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:28.297639 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:28.297718 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:28.298374 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:28.298376 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:28.298391 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:28.298479 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.298486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.299790 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.299793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.301052 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.301064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.303101 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.303104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.303215 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.303239 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 06:40:28.303301 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 06:40:28.303367 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":12,"thread_start_us":74,"threads_started":1}
I20260928 06:40:28.303709 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.296933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.8.151.195:41223)
0928 06:40:28.297861 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:28.297871 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:28.297889 (+ 18us) server_negotiation.cc:407] Connection header received
0928 06:40:28.297950 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:28.297954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:28.298105 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:28.298226 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:28.298444 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.299610 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.301157 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.301927 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.302078 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.303372 (+ 1294us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:28.303421 (+ 49us) server_negotiation.cc:299] Negotiation successful
0928 06:40:28.303505 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":145,"threads_started":1}
I20260928 06:40:28.304239 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.297449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46542 (local address 127.8.151.194:39039)
0928 06:40:28.297753 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:28.297764 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:28.297786 (+ 22us) server_negotiation.cc:407] Connection header received
0928 06:40:28.297857 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:28.297862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:28.298078 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:28.298219 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:28.298613 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.299465 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.302211 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.302970 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.303080 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.303945 (+ 865us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:28.303996 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 06:40:28.304095 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":120,"threads_started":1}
I20260928 06:40:28.305051 13022 tablet_service.cc:1467] Processing CreateTablet for tablet 90274caebbd247039164a624fc64510e (DEFAULT_TABLE table=test-table [id=aa6a8f867330480e85c529e97b1ea1cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:28.306214 13022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90274caebbd247039164a624fc64510e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:28.306968 13155 tablet_service.cc:1467] Processing CreateTablet for tablet 90274caebbd247039164a624fc64510e (DEFAULT_TABLE table=test-table [id=aa6a8f867330480e85c529e97b1ea1cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:28.308010 13155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90274caebbd247039164a624fc64510e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:28.309311 13288 tablet_service.cc:1467] Processing CreateTablet for tablet 90274caebbd247039164a624fc64510e (DEFAULT_TABLE table=test-table [id=aa6a8f867330480e85c529e97b1ea1cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:28.310089 13288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90274caebbd247039164a624fc64510e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:28.313380 13371 tablet_bootstrap.cc:492] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669: Bootstrap starting.
I20260928 06:40:28.315120 13372 tablet_bootstrap.cc:492] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: Bootstrap starting.
I20260928 06:40:28.315729 13371 tablet_bootstrap.cc:654] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:28.316365 13373 tablet_bootstrap.cc:492] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: Bootstrap starting.
I20260928 06:40:28.316992 13372 tablet_bootstrap.cc:654] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:28.316975 13371 log.cc:824] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:28.318140 13372 log.cc:824] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:28.318545 13373 tablet_bootstrap.cc:654] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:28.318784 13371 tablet_bootstrap.cc:492] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669: No bootstrap required, opened a new log
I20260928 06:40:28.318970 13371 ts_tablet_manager.cc:1397] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 06:40:28.319710 13373 log.cc:824] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:28.320173 13372 tablet_bootstrap.cc:492] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: No bootstrap required, opened a new log
I20260928 06:40:28.320339 13372 ts_tablet_manager.cc:1397] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260928 06:40:28.321560 13373 tablet_bootstrap.cc:492] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: No bootstrap required, opened a new log
I20260928 06:40:28.321776 13373 ts_tablet_manager.cc:1397] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260928 06:40:28.322328 13371 raft_consensus.cc:348] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236aba066969449ea3d2ba5e9ac29669" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45071 } } peers { permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 } } peers { permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 } }
I20260928 06:40:28.322551 13371 raft_consensus.cc:374] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:28.322607 13371 raft_consensus.cc:729] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236aba066969449ea3d2ba5e9ac29669, State: Initialized, Role: FOLLOWER
I20260928 06:40:28.323007 13371 consensus_queue.cc:260] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236aba066969449ea3d2ba5e9ac29669" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45071 } } peers { permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 } } peers { permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 } }
I20260928 06:40:28.323634 13088 heartbeater.cc:503] Master 127.8.151.254:35793 was elected leader, sending a full tablet report...
I20260928 06:40:28.324009 13372 raft_consensus.cc:348] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236aba066969449ea3d2ba5e9ac29669" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45071 } } peers { permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 } } peers { permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 } }
I20260928 06:40:28.324291 13372 raft_consensus.cc:374] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:28.324343 13372 raft_consensus.cc:729] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cee44e363214cdaaa83d79b218b7400, State: Initialized, Role: FOLLOWER
I20260928 06:40:28.324802 13372 consensus_queue.cc:260] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236aba066969449ea3d2ba5e9ac29669" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45071 } } peers { permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 } } peers { permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 } }
I20260928 06:40:28.324899 13371 ts_tablet_manager.cc:1428] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260928 06:40:28.325503 13221 heartbeater.cc:503] Master 127.8.151.254:35793 was elected leader, sending a full tablet report...
I20260928 06:40:28.325381 13373 raft_consensus.cc:348] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236aba066969449ea3d2ba5e9ac29669" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45071 } } peers { permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 } } peers { permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 } }
I20260928 06:40:28.325654 13373 raft_consensus.cc:374] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:28.325702 13373 raft_consensus.cc:729] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf1922a8af948cfb79c0a57e645582e, State: Initialized, Role: FOLLOWER
I20260928 06:40:28.325734 13372 ts_tablet_manager.cc:1428] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260928 06:40:28.326140 13373 consensus_queue.cc:260] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236aba066969449ea3d2ba5e9ac29669" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45071 } } peers { permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 } } peers { permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 } }
I20260928 06:40:28.327001 13354 heartbeater.cc:503] Master 127.8.151.254:35793 was elected leader, sending a full tablet report...
I20260928 06:40:28.327035 13373 ts_tablet_manager.cc:1428] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
W20260928 06:40:28.338744 13089 tablet.cc:2369] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 06:40:28.500116 13355 tablet.cc:2369] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 06:40:28.539572 13222 tablet.cc:2369] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:28.635000 13377 raft_consensus.cc:482] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:28.635241 13377 raft_consensus.cc:504] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236aba066969449ea3d2ba5e9ac29669" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45071 } } peers { permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 } } peers { permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 } }
I20260928 06:40:28.636488 13377 leader_election.cc:288] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf1922a8af948cfb79c0a57e645582e (127.8.151.195:41223), 4cee44e363214cdaaa83d79b218b7400 (127.8.151.194:39039)
I20260928 06:40:28.640362 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.636859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:40141 (local address 127.8.151.195:41223)
0928 06:40:28.637010 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:28.637015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:28.637097 (+ 82us) server_negotiation.cc:407] Connection header received
0928 06:40:28.637265 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:28.637268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:28.637318 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:28.637392 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:28.637708 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.638311 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.639422 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.639843 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.639935 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.639973 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:28.640172 (+ 199us) server_negotiation.cc:299] Negotiation successful
0928 06:40:28.640239 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 06:40:28.640363 13380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.636761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:41223 (local address 127.8.151.193:40141)
0928 06:40:28.637013 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:28.637027 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:28.637156 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:28.637424 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:28.637428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:28.637453 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:28.637594 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.637600 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.638415 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.638419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.639310 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.639320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.639955 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.639959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.640051 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.640080 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 06:40:28.640147 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 06:40:28.640211 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260928 06:40:28.640424 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.637250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:53011 (local address 127.8.151.194:39039)
0928 06:40:28.637384 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:28.637389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:28.637488 (+ 99us) server_negotiation.cc:407] Connection header received
0928 06:40:28.637584 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:28.637587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:28.637631 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:28.637702 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:28.637935 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.638458 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.639537 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.639942 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.640047 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.640095 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:28.640265 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 06:40:28.640325 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 06:40:28.640425 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.637117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:39039 (local address 127.8.151.193:53011)
0928 06:40:28.637412 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:28.637423 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:28.637506 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:28.637715 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:28.637718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:28.637734 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:28.637840 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.637845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.638560 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.638562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.639424 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:28.639434 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.640055 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.640058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:28.640149 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.640171 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 06:40:28.640240 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 06:40:28.640303 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":92,"threads_started":1}
I20260928 06:40:28.640962 13175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90274caebbd247039164a624fc64510e" candidate_uuid: "236aba066969449ea3d2ba5e9ac29669" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cee44e363214cdaaa83d79b218b7400" is_pre_election: true
I20260928 06:40:28.640969 13308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90274caebbd247039164a624fc64510e" candidate_uuid: "236aba066969449ea3d2ba5e9ac29669" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf1922a8af948cfb79c0a57e645582e" is_pre_election: true
I20260928 06:40:28.641273 13308 raft_consensus.cc:2454] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 236aba066969449ea3d2ba5e9ac29669 in term 0.
I20260928 06:40:28.641273 13175 raft_consensus.cc:2454] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 236aba066969449ea3d2ba5e9ac29669 in term 0.
I20260928 06:40:28.641728 12977 leader_election.cc:302] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236aba066969449ea3d2ba5e9ac29669, 4cee44e363214cdaaa83d79b218b7400; no voters:
I20260928 06:40:28.641949 13377 raft_consensus.cc:2781] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:28.642027 13377 raft_consensus.cc:482] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:28.642056 13377 raft_consensus.cc:3037] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:28.643018 13377 raft_consensus.cc:504] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236aba066969449ea3d2ba5e9ac29669" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45071 } } peers { permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 } } peers { permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 } }
I20260928 06:40:28.643368 13377 leader_election.cc:288] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf1922a8af948cfb79c0a57e645582e (127.8.151.195:41223), 4cee44e363214cdaaa83d79b218b7400 (127.8.151.194:39039)
I20260928 06:40:28.643676 13308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90274caebbd247039164a624fc64510e" candidate_uuid: "236aba066969449ea3d2ba5e9ac29669" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf1922a8af948cfb79c0a57e645582e"
I20260928 06:40:28.643774 13175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90274caebbd247039164a624fc64510e" candidate_uuid: "236aba066969449ea3d2ba5e9ac29669" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cee44e363214cdaaa83d79b218b7400"
I20260928 06:40:28.643796 13308 raft_consensus.cc:3037] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:28.643900 13175 raft_consensus.cc:3037] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:28.644783 13308 raft_consensus.cc:2454] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 236aba066969449ea3d2ba5e9ac29669 in term 1.
I20260928 06:40:28.644888 13175 raft_consensus.cc:2454] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 236aba066969449ea3d2ba5e9ac29669 in term 1.
I20260928 06:40:28.645059 12977 leader_election.cc:302] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236aba066969449ea3d2ba5e9ac29669, 5bf1922a8af948cfb79c0a57e645582e; no voters:
I20260928 06:40:28.645231 13377 raft_consensus.cc:2781] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:28.645430 13377 raft_consensus.cc:686] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 1 LEADER]: Becoming Leader. State: Replica: 236aba066969449ea3d2ba5e9ac29669, State: Running, Role: LEADER
I20260928 06:40:28.645752 13377 consensus_queue.cc:237] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236aba066969449ea3d2ba5e9ac29669" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45071 } } peers { permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 } } peers { permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 } }
I20260928 06:40:28.648711 12898 catalog_manager.cc:5495] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 reported cstate change: term changed from 0 to 1, leader changed from <none> to 236aba066969449ea3d2ba5e9ac29669 (127.8.151.193). New cstate: current_term: 1 leader_uuid: "236aba066969449ea3d2ba5e9ac29669" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236aba066969449ea3d2ba5e9ac29669" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 45071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 } health_report { overall_health: UNKNOWN } } }
I20260928 06:40:28.666975 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.663698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38530 (local address 127.8.151.193:45071)
0928 06:40:28.663829 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:28.663835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:28.663847 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:28.663917 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:28.663919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:28.663969 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:28.664055 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:28.664300 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.664803 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.665635 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:28.665853 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:28.665937 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:28.666212 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:28.666619 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:28.666735 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:28.666789 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 06:40:28.666843 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 06:40:28.672756 13175 raft_consensus.cc:1261] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400 [term 1 FOLLOWER]: Refusing update from remote peer 236aba066969449ea3d2ba5e9ac29669: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:28.672756 13308 raft_consensus.cc:1261] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e [term 1 FOLLOWER]: Refusing update from remote peer 236aba066969449ea3d2ba5e9ac29669: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:28.673406 13382 consensus_queue.cc:1035] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cee44e363214cdaaa83d79b218b7400" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:28.673600 13377 consensus_queue.cc:1035] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf1922a8af948cfb79c0a57e645582e" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:28.681025 13386 mvcc.cc:205] Tried to move back new op lower bound from 7334205967036239872 to 7334205966935371776. Current Snapshot: MvccSnapshot[applied={T|T < 7334205967036239872}]
I20260928 06:40:28.682039 13387 mvcc.cc:205] Tried to move back new op lower bound from 7334205967036239872 to 7334205966935371776. Current Snapshot: MvccSnapshot[applied={T|T < 7334205967036239872}]
I20260928 06:40:28.688405 12898 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:28.688560 12898 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:28.690873 12898 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bcc4c0548ceb44b9ac61e09ef5552293: Sending DeleteTablet for 3 replicas of tablet 90274caebbd247039164a624fc64510e
I20260928 06:40:28.691650 13022 tablet_service.cc:1514] Processing DeleteTablet for tablet 90274caebbd247039164a624fc64510e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:28 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:38526
I20260928 06:40:28.691782 13288 tablet_service.cc:1514] Processing DeleteTablet for tablet 90274caebbd247039164a624fc64510e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:28 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48290
I20260928 06:40:28.691780 13155 tablet_service.cc:1514] Processing DeleteTablet for tablet 90274caebbd247039164a624fc64510e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:28 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:46542
I20260928 06:40:28.692235 13396 tablet_replica.cc:330] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: stopping tablet replica
I20260928 06:40:28.692417 13394 tablet_replica.cc:330] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669: stopping tablet replica
I20260928 06:40:28.692556 13396 raft_consensus.cc:2229] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:28.692734 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 12960
I20260928 06:40:28.692860 13394 raft_consensus.cc:2229] T 90274caebbd247039164a624fc64510e P 236aba066969449ea3d2ba5e9ac29669 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:28.693327 13395 tablet_replica.cc:330] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: stopping tablet replica
I20260928 06:40:28.693434 13396 raft_consensus.cc:2258] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:28.693686 13395 raft_consensus.cc:2229] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:28.693987 13395 raft_consensus.cc:2258] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:28.694980 13396 ts_tablet_manager.cc:1905] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:28.695475 13395 ts_tablet_manager.cc:1905] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:28.697161 13396 ts_tablet_manager.cc:1918] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:28.697245 13396 log.cc:1196] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/90274caebbd247039164a624fc64510e
I20260928 06:40:28.697513 13396 ts_tablet_manager.cc:1939] T 90274caebbd247039164a624fc64510e P 4cee44e363214cdaaa83d79b218b7400: Deleting consensus metadata
I20260928 06:40:28.697891 13395 ts_tablet_manager.cc:1918] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:28.697983 13395 log.cc:1196] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/90274caebbd247039164a624fc64510e
I20260928 06:40:28.698271 13395 ts_tablet_manager.cc:1939] T 90274caebbd247039164a624fc64510e P 5bf1922a8af948cfb79c0a57e645582e: Deleting consensus metadata
I20260928 06:40:28.698598 12884 catalog_manager.cc:4841] TS 4cee44e363214cdaaa83d79b218b7400 (127.8.151.194:39039): tablet 90274caebbd247039164a624fc64510e (table test-table [id=aa6a8f867330480e85c529e97b1ea1cc]) successfully deleted
I20260928 06:40:28.699297 12884 catalog_manager.cc:4841] TS 5bf1922a8af948cfb79c0a57e645582e (127.8.151.195:41223): tablet 90274caebbd247039164a624fc64510e (table test-table [id=aa6a8f867330480e85c529e97b1ea1cc]) successfully deleted
W20260928 06:40:28.699815 12884 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:45071 (error 108)
I20260928 06:40:28.699986 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 13093
I20260928 06:40:28.702042 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.701794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:45071 (local address 127.0.0.1:38532)
0928 06:40:28.701910 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:28.701972 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 06:40:28.702270 12884 catalog_manager.cc:4584] TS 236aba066969449ea3d2ba5e9ac29669 (127.8.151.193:45071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90274caebbd247039164a624fc64510e: Network error: Client connection negotiation failed: client connection to 127.8.151.193:45071: connect: Connection refused (error 111)
I20260928 06:40:28.706861 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 13226
I20260928 06:40:28.711321 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 12867
2026-09-28T06:40:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2370 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-28T06:40:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:28Z Disabled control of system clock
I20260928 06:40:28.739957 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:35027
--webserver_interface=127.8.151.254
--webserver_port=0
--builtin_ntp_servers=127.8.151.212:39751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:35027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 06:40:28.844894 13401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:28.845139 13401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:28.845172 13401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:28.849025 13401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:28.849071 13401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:28.849083 13401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:28.849098 13401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:28.849112 13401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:28.853714 13401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:39751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:35027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:35027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:28.854775 13401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:28.856153 13401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:28.864408 13409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:28.864532 13401 server_base.cc:1034] running on GCE node
W20260928 06:40:28.864396 13407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:28.864415 13406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:28.865145 13401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:28.866231 13401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:28.867385 13401 hybrid_clock.cc:648] HybridClock initialized: now 1790577628867362 us; error 37 us; skew 500 ppm
I20260928 06:40:28.869135 13401 webserver.cc:460] Webserver started at http://127.8.151.254:45659/ using document root <none> and password file <none>
I20260928 06:40:28.869681 13401 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:28.869748 13401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:28.869943 13401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:28.871965 13401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "171faf464e5a4e5e8e2208dc5c70a3fb"
format_stamp: "Formatted at 2026-09-28 06:40:28 on dist-test-slave-gtr8"
server_key: "ed3b579b9c8f18a00a4666dc4e7aa287"
server_key_iv: "b66e7435b41f36fb0d4011d404284890"
server_key_version: "encryptionkey@0"
I20260928 06:40:28.872479 13401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "171faf464e5a4e5e8e2208dc5c70a3fb"
format_stamp: "Formatted at 2026-09-28 06:40:28 on dist-test-slave-gtr8"
server_key: "ed3b579b9c8f18a00a4666dc4e7aa287"
server_key_iv: "b66e7435b41f36fb0d4011d404284890"
server_key_version: "encryptionkey@0"
I20260928 06:40:28.875962 13401 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:40:28.878224 13415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:28.879437 13401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:40:28.879545 13401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "171faf464e5a4e5e8e2208dc5c70a3fb"
format_stamp: "Formatted at 2026-09-28 06:40:28 on dist-test-slave-gtr8"
server_key: "ed3b579b9c8f18a00a4666dc4e7aa287"
server_key_iv: "b66e7435b41f36fb0d4011d404284890"
server_key_version: "encryptionkey@0"
I20260928 06:40:28.879634 13401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:28.883962 13401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:28.884670 13401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:28.884838 13401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:28.891615 13401 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:35027
I20260928 06:40:28.891625 13467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:35027 every 8 connection(s)
I20260928 06:40:28.892490 13401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:28.894889 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 13401
I20260928 06:40:28.894986 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:28.895306 8799 external_mini_cluster.cc:1428] Setting key c7117db1b6a5328a206c4cf6645088ad
I20260928 06:40:28.895620 13468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:28.901011 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:28.896657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33876 (local address 127.8.151.254:35027)
0928 06:40:28.897327 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:28.897348 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:28.897383 (+ 35us) server_negotiation.cc:407] Connection header received
0928 06:40:28.898286 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:28.898306 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:28.898648 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:28.898964 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:28.899784 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:28.899788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:28.899800 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:28.899809 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:28.899863 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:28.899879 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:28.900008 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:28.900074 (+ 66us) server_negotiation.cc:299] Negotiation successful
0928 06:40:28.900330 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":138,"threads_started":1}
I20260928 06:40:28.902328 13468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb: Bootstrap starting.
I20260928 06:40:28.904567 13468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:28.905365 13468 log.cc:824] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:28.907354 13468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb: No bootstrap required, opened a new log
I20260928 06:40:28.910900 13468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171faf464e5a4e5e8e2208dc5c70a3fb" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } }
I20260928 06:40:28.911129 13468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:28.911204 13468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 171faf464e5a4e5e8e2208dc5c70a3fb, State: Initialized, Role: FOLLOWER
I20260928 06:40:28.911582 13468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171faf464e5a4e5e8e2208dc5c70a3fb" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } }
I20260928 06:40:28.911708 13468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:28.911747 13468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:28.911816 13468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:28.912703 13468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171faf464e5a4e5e8e2208dc5c70a3fb" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } }
I20260928 06:40:28.913012 13468 leader_election.cc:302] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 171faf464e5a4e5e8e2208dc5c70a3fb; no voters:
I20260928 06:40:28.913281 13468 leader_election.cc:288] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:28.913513 13473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:28.913933 13473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [term 1 LEADER]: Becoming Leader. State: Replica: 171faf464e5a4e5e8e2208dc5c70a3fb, State: Running, Role: LEADER
I20260928 06:40:28.914381 13473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171faf464e5a4e5e8e2208dc5c70a3fb" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } }
I20260928 06:40:28.914541 13468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:28.916100 13474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "171faf464e5a4e5e8e2208dc5c70a3fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171faf464e5a4e5e8e2208dc5c70a3fb" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } } }
I20260928 06:40:28.916101 13475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 171faf464e5a4e5e8e2208dc5c70a3fb. Latest consensus state: current_term: 1 leader_uuid: "171faf464e5a4e5e8e2208dc5c70a3fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171faf464e5a4e5e8e2208dc5c70a3fb" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } } }
I20260928 06:40:28.916280 13475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:28.916280 13474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb [sys.catalog]: This master's current role is: LEADER
W20260928 06:40:28.919363 13488 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:40:28.919443 13488 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:40:28.919538 13487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:28.920290 13487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:28.925380 13487 catalog_manager.cc:1347] Generated new cluster ID: 560276bf2d374155bcb118f0f7a58c0e
I20260928 06:40:28.925446 13487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:28.946532 13487 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:28.946869 13487 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:28.953657 13487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb: Generated new TSK 0
I20260928 06:40:28.954422 13487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:40:28.958279 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--builtin_ntp_servers=127.8.151.212:39751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 06:40:29.065040 13492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:29.065304 13492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:29.065361 13492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:29.069319 13492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:29.069379 13492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:29.069435 13492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:29.074065 13492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:39751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:29.075039 13492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:29.076388 13492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:29.085775 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:29.085755 13498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:29.085845 13492 server_base.cc:1034] running on GCE node
W20260928 06:40:29.085870 13497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:29.086403 13492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:29.087047 13492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:29.088202 13492 hybrid_clock.cc:648] HybridClock initialized: now 1790577629088184 us; error 33 us; skew 500 ppm
I20260928 06:40:29.090029 13492 webserver.cc:460] Webserver started at http://127.8.151.193:40721/ using document root <none> and password file <none>
I20260928 06:40:29.090586 13492 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:29.090653 13492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:29.090864 13492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:29.092950 13492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "5c1ae984a5574323a72999e5510b1f4c"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "8c5ea8758d866a1b6f54c5e8655b1d0f"
server_key_iv: "cd58a11c0e0d986ab9484fe8d76aa65c"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.093479 13492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "5c1ae984a5574323a72999e5510b1f4c"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "8c5ea8758d866a1b6f54c5e8655b1d0f"
server_key_iv: "cd58a11c0e0d986ab9484fe8d76aa65c"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.097123 13492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:29.099391 13506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:29.100593 13492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:40:29.100703 13492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "5c1ae984a5574323a72999e5510b1f4c"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "8c5ea8758d866a1b6f54c5e8655b1d0f"
server_key_iv: "cd58a11c0e0d986ab9484fe8d76aa65c"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.100809 13492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:29.105170 13492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:29.105732 13492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:29.105875 13492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:29.106375 13492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:29.107234 13492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:29.107280 13492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:29.107328 13492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:29.107355 13492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:29.115746 13492 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:33749
I20260928 06:40:29.115908 13619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:33749 every 8 connection(s)
I20260928 06:40:29.116652 13492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:29.122526 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.118418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:43221 (local address 127.8.151.254:35027)
0928 06:40:29.118558 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.118562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.119639 (+ 1077us) server_negotiation.cc:407] Connection header received
0928 06:40:29.120909 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.120912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.120956 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.121031 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.122100 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.122103 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.122106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.122119 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.122141 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.122149 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.122226 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.122343 (+ 117us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.122388 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:40:29.123415 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.118818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:35027 (local address 127.8.151.193:43221)
0928 06:40:29.119501 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:29.119540 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:29.120713 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:29.121205 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:29.121213 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:29.121299 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:29.121846 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:40:29.121848 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:29.121890 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:40:29.121957 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:29.122217 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:29.122226 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:29.122231 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 06:40:29.122393 (+ 162us) client_negotiation.cc:240] Negotiation successful
0928 06:40:29.122634 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":102,"threads_started":1}
I20260928 06:40:29.124262 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 13492
I20260928 06:40:29.124374 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:29.124670 8799 external_mini_cluster.cc:1428] Setting key a674825fa7ac4031457eefc24f713725
I20260928 06:40:29.124711 13620 heartbeater.cc:347] Connected to a master server at 127.8.151.254:35027
I20260928 06:40:29.124934 13620 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:29.125381 13620 heartbeater.cc:511] Master 127.8.151.254:35027 requested a full tablet report, sending...
I20260928 06:40:29.126652 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--builtin_ntp_servers=127.8.151.212:39751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260928 06:40:29.126888 13426 ts_manager.cc:194] Registered new tserver with Master: 5c1ae984a5574323a72999e5510b1f4c (127.8.151.193:33749)
W20260928 06:40:29.233423 13623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:29.233709 13623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:29.233791 13623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:29.237937 13623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:29.238003 13623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:29.238081 13623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:29.242843 13623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:39751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:29.243935 13623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:29.245270 13623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:29.254494 13629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:29.254484 13628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:29.254581 13631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:29.254712 13623 server_base.cc:1034] running on GCE node
I20260928 06:40:29.255115 13623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:29.255775 13623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:29.256940 13623 hybrid_clock.cc:648] HybridClock initialized: now 1790577629256918 us; error 33 us; skew 500 ppm
I20260928 06:40:29.258735 13623 webserver.cc:460] Webserver started at http://127.8.151.194:40489/ using document root <none> and password file <none>
I20260928 06:40:29.259317 13623 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:29.259385 13623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:29.259573 13623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:29.261560 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "8be4eec7ffe94d98a0b8060fc93dfabd"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "b54900a020d09b883de7e3ff6f94b8c2"
server_key_iv: "30986846c9b9530fbe195027e117073d"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.262084 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "8be4eec7ffe94d98a0b8060fc93dfabd"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "b54900a020d09b883de7e3ff6f94b8c2"
server_key_iv: "30986846c9b9530fbe195027e117073d"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.265532 13623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:29.267879 13637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:29.269038 13623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:29.269138 13623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "8be4eec7ffe94d98a0b8060fc93dfabd"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "b54900a020d09b883de7e3ff6f94b8c2"
server_key_iv: "30986846c9b9530fbe195027e117073d"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.269237 13623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:29.273603 13623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:29.274165 13623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:29.274313 13623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:29.274832 13623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:29.275705 13623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:29.275749 13623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:29.275797 13623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:29.275825 13623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:29.283769 13623 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:45519
I20260928 06:40:29.283797 13750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:45519 every 8 connection(s)
I20260928 06:40:29.284590 13623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:29.290270 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.286337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:43851 (local address 127.8.151.254:35027)
0928 06:40:29.286474 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.286477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.287503 (+ 1026us) server_negotiation.cc:407] Connection header received
0928 06:40:29.288721 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.288724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.288766 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.288839 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.289836 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.289840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.289842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.289854 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.289876 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.289884 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.289960 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.290073 (+ 113us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.290115 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:40:29.291028 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.286694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:35027 (local address 127.8.151.194:43851)
0928 06:40:29.287370 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:29.287410 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:29.288524 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:29.289001 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:29.289007 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:29.289093 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:29.289596 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:40:29.289597 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:29.289635 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:40:29.289701 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:29.289950 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:29.289957 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:29.289962 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 06:40:29.290109 (+ 147us) client_negotiation.cc:240] Negotiation successful
0928 06:40:29.290346 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":96,"threads_started":1}
I20260928 06:40:29.292032 13751 heartbeater.cc:347] Connected to a master server at 127.8.151.254:35027
I20260928 06:40:29.292233 13751 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:29.292529 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 13623
I20260928 06:40:29.292601 13751 heartbeater.cc:511] Master 127.8.151.254:35027 requested a full tablet report, sending...
I20260928 06:40:29.292627 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:29.292874 8799 external_mini_cluster.cc:1428] Setting key 9f632a8a0afab1a217cdc9d545be92e8
I20260928 06:40:29.293550 13426 ts_manager.cc:194] Registered new tserver with Master: 8be4eec7ffe94d98a0b8060fc93dfabd (127.8.151.194:45519)
I20260928 06:40:29.294790 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--builtin_ntp_servers=127.8.151.212:39751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 06:40:29.401681 13754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:29.401937 13754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:29.401989 13754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:29.406056 13754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:29.406129 13754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:29.406203 13754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:29.410868 13754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:39751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:29.411940 13754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:29.413275 13754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:29.422478 13759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:29.422451 13760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:29.422561 13754 server_base.cc:1034] running on GCE node
W20260928 06:40:29.422464 13762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:29.423130 13754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:29.423794 13754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:29.424940 13754 hybrid_clock.cc:648] HybridClock initialized: now 1790577629424914 us; error 39 us; skew 500 ppm
I20260928 06:40:29.426610 13754 webserver.cc:460] Webserver started at http://127.8.151.195:42761/ using document root <none> and password file <none>
I20260928 06:40:29.427183 13754 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:29.427261 13754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:29.427459 13754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:29.429432 13754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "df60aae7503e4bf899583f2db4ad41cf"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "ba8538c566f61d0506bd27549812a526"
server_key_iv: "ba8cbbb0cf0d397513cb097cdc6ac14f"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.429956 13754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "df60aae7503e4bf899583f2db4ad41cf"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "ba8538c566f61d0506bd27549812a526"
server_key_iv: "ba8cbbb0cf0d397513cb097cdc6ac14f"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.433594 13754 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:29.436007 13768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:29.437249 13754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:29.437351 13754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "df60aae7503e4bf899583f2db4ad41cf"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "ba8538c566f61d0506bd27549812a526"
server_key_iv: "ba8cbbb0cf0d397513cb097cdc6ac14f"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.437464 13754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:29.441903 13754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:29.442477 13754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:29.442621 13754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:29.443120 13754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:29.444018 13754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:29.444063 13754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:29.444111 13754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:29.444140 13754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:29.452140 13754 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:35019
I20260928 06:40:29.452188 13881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:35019 every 8 connection(s)
I20260928 06:40:29.452966 13754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:29.458657 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.454697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:34841 (local address 127.8.151.254:35027)
0928 06:40:29.454865 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.454869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.455974 (+ 1105us) server_negotiation.cc:407] Connection header received
0928 06:40:29.457203 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.457205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.457246 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.457318 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.458349 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.458351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.458353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.458360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.458374 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.458380 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.458436 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.458516 (+ 80us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.458550 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 06:40:29.459511 13884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.455172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:35027 (local address 127.8.151.195:34841)
0928 06:40:29.455823 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:29.455861 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:29.457001 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:29.457460 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:29.457467 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:29.457553 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:29.458082 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:40:29.458083 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:29.458158 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:40:29.458231 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:29.458426 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:29.458430 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:29.458434 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 06:40:29.458545 (+ 111us) client_negotiation.cc:240] Negotiation successful
0928 06:40:29.458739 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":95,"threads_started":1}
I20260928 06:40:29.460394 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 13754
I20260928 06:40:29.460500 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:29.460546 13882 heartbeater.cc:347] Connected to a master server at 127.8.151.254:35027
I20260928 06:40:29.460783 13882 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:29.460798 8799 external_mini_cluster.cc:1428] Setting key 90af12ef4cdc372f2c970d7eb2388f0c
I20260928 06:40:29.461220 13882 heartbeater.cc:511] Master 127.8.151.254:35027 requested a full tablet report, sending...
I20260928 06:40:29.462195 13426 ts_manager.cc:194] Registered new tserver with Master: df60aae7503e4bf899583f2db4ad41cf (127.8.151.195:35019)
I20260928 06:40:29.474644 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:40:29.478017 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.476498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33886 (local address 127.8.151.254:35027)
0928 06:40:29.476610 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.476616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.476798 (+ 182us) server_negotiation.cc:407] Connection header received
0928 06:40:29.476919 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.476921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.476955 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.477014 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.477665 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.477668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.477671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.477683 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.477703 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.477711 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.477778 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.477850 (+ 72us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.477892 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 06:40:29.480618 13426 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:29.483680 13426 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:29.495278 13895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.491847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:45519 (local address 127.0.0.1:54882)
0928 06:40:29.492804 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:29.492816 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:29.492889 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:29.493876 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:29.493882 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:29.493958 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:29.494490 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:40:29.494492 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:29.494514 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:40:29.494584 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:29.494942 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:29.494951 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:29.494957 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:40:29.495043 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 06:40:29.495108 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":90,"threads_started":1}
I20260928 06:40:29.495311 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.491627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:35019 (local address 127.0.0.1:41408)
0928 06:40:29.492640 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:29.492652 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:29.492727 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:29.493875 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:29.493882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:29.493957 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:29.494723 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:40:29.494725 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:29.494734 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:40:29.494756 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:29.495109 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:29.495112 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:29.495115 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 06:40:29.495200 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 06:40:29.495228 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"thread_start_us":107,"threads_started":1}
I20260928 06:40:29.495716 13897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.492250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.8.151.195:35019)
0928 06:40:29.492630 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.492643 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.492820 (+ 177us) server_negotiation.cc:407] Connection header received
0928 06:40:29.492869 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.492874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.493089 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.493218 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.494894 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.494900 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.494911 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.494922 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.494961 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.494988 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.495074 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.495475 (+ 401us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.495549 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":93,"threads_started":1}
I20260928 06:40:29.495875 13898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.492286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54882 (local address 127.8.151.194:45519)
0928 06:40:29.492751 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.492763 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.492897 (+ 134us) server_negotiation.cc:407] Connection header received
0928 06:40:29.492947 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.492952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.493089 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.493219 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.494728 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.494735 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.494747 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.494759 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.494818 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.494851 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.495622 (+ 771us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.495666 (+ 44us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.495730 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":108,"threads_started":1}
I20260928 06:40:29.496078 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.491514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:33749 (local address 127.0.0.1:49348)
0928 06:40:29.492194 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:29.492266 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:29.492364 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:29.494135 (+ 1771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:29.494139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:29.494154 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:29.494589 (+ 435us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:40:29.494591 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:29.494627 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:40:29.494657 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:29.495191 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:29.495194 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:29.495198 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 06:40:29.495296 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 06:40:29.495900 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":68,"threads_started":1}
I20260928 06:40:29.496479 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.491635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49348 (local address 127.8.151.193:33749)
0928 06:40:29.493103 (+ 1468us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.493115 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.493138 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:40:29.493189 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.493194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.493376 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.493505 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.494879 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.494910 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.494925 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.494938 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.495010 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.495049 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.496220 (+ 1171us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.496266 (+ 46us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.496327 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1369,"thread_start_us":93,"threads_started":1}
I20260928 06:40:29.499043 13685 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9a6cbba3dc432e917d3962b056fdb2 (DEFAULT_TABLE table=test-table [id=de1b712d6daf493eaa00238db0f677f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:29.499186 13816 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9a6cbba3dc432e917d3962b056fdb2 (DEFAULT_TABLE table=test-table [id=de1b712d6daf493eaa00238db0f677f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:29.500175 13816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9a6cbba3dc432e917d3962b056fdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:29.500178 13685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9a6cbba3dc432e917d3962b056fdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:29.500496 13554 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9a6cbba3dc432e917d3962b056fdb2 (DEFAULT_TABLE table=test-table [id=de1b712d6daf493eaa00238db0f677f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:29.501389 13554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9a6cbba3dc432e917d3962b056fdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:29.506029 13899 tablet_bootstrap.cc:492] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: Bootstrap starting.
I20260928 06:40:29.507328 13900 tablet_bootstrap.cc:492] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: Bootstrap starting.
I20260928 06:40:29.507890 13901 tablet_bootstrap.cc:492] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c: Bootstrap starting.
I20260928 06:40:29.508303 13899 tablet_bootstrap.cc:654] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:29.509125 13899 log.cc:824] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:29.509322 13900 tablet_bootstrap.cc:654] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:29.509734 13901 tablet_bootstrap.cc:654] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:29.510102 13900 log.cc:824] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:29.510454 13901 log.cc:824] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:29.511245 13899 tablet_bootstrap.cc:492] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: No bootstrap required, opened a new log
I20260928 06:40:29.511426 13899 ts_tablet_manager.cc:1397] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:40:29.511962 13900 tablet_bootstrap.cc:492] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: No bootstrap required, opened a new log
I20260928 06:40:29.512085 13901 tablet_bootstrap.cc:492] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c: No bootstrap required, opened a new log
I20260928 06:40:29.512130 13900 ts_tablet_manager.cc:1397] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260928 06:40:29.512257 13901 ts_tablet_manager.cc:1397] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260928 06:40:29.515759 13899 raft_consensus.cc:348] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c1ae984a5574323a72999e5510b1f4c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33749 } } peers { permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 } } peers { permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 } }
I20260928 06:40:29.516031 13899 raft_consensus.cc:374] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:29.516086 13899 raft_consensus.cc:729] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df60aae7503e4bf899583f2db4ad41cf, State: Initialized, Role: FOLLOWER
I20260928 06:40:29.516000 13901 raft_consensus.cc:348] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c1ae984a5574323a72999e5510b1f4c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33749 } } peers { permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 } } peers { permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 } }
I20260928 06:40:29.516252 13901 raft_consensus.cc:374] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:29.516312 13901 raft_consensus.cc:729] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1ae984a5574323a72999e5510b1f4c, State: Initialized, Role: FOLLOWER
I20260928 06:40:29.516183 13900 raft_consensus.cc:348] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c1ae984a5574323a72999e5510b1f4c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33749 } } peers { permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 } } peers { permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 } }
I20260928 06:40:29.516388 13900 raft_consensus.cc:374] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:29.516443 13900 raft_consensus.cc:729] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be4eec7ffe94d98a0b8060fc93dfabd, State: Initialized, Role: FOLLOWER
I20260928 06:40:29.516558 13899 consensus_queue.cc:260] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c1ae984a5574323a72999e5510b1f4c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33749 } } peers { permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 } } peers { permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 } }
I20260928 06:40:29.516734 13901 consensus_queue.cc:260] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c1ae984a5574323a72999e5510b1f4c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33749 } } peers { permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 } } peers { permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 } }
I20260928 06:40:29.516814 13900 consensus_queue.cc:260] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c1ae984a5574323a72999e5510b1f4c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33749 } } peers { permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 } } peers { permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 } }
I20260928 06:40:29.517323 13620 heartbeater.cc:503] Master 127.8.151.254:35027 was elected leader, sending a full tablet report...
I20260928 06:40:29.517560 13899 ts_tablet_manager.cc:1428] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260928 06:40:29.517560 13900 ts_tablet_manager.cc:1428] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260928 06:40:29.517894 13882 heartbeater.cc:503] Master 127.8.151.254:35027 was elected leader, sending a full tablet report...
I20260928 06:40:29.518061 13751 heartbeater.cc:503] Master 127.8.151.254:35027 was elected leader, sending a full tablet report...
I20260928 06:40:29.519441 13901 ts_tablet_manager.cc:1428] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260928 06:40:29.535450 13752 tablet.cc:2369] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 06:40:29.617791 13621 tablet.cc:2369] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:29.691495 13906 raft_consensus.cc:482] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:29.691710 13906 raft_consensus.cc:504] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c1ae984a5574323a72999e5510b1f4c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33749 } } peers { permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 } } peers { permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 } }
I20260928 06:40:29.692850 13906 leader_election.cc:288] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df60aae7503e4bf899583f2db4ad41cf (127.8.151.195:35019), 8be4eec7ffe94d98a0b8060fc93dfabd (127.8.151.194:45519)
I20260928 06:40:29.694805 13897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.693193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:41215 (local address 127.8.151.195:35019)
0928 06:40:29.693320 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.693324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.693530 (+ 206us) server_negotiation.cc:407] Connection header received
0928 06:40:29.693618 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.693621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.693665 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.693742 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.694440 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.694444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.694445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.694457 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.694475 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.694483 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.694555 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.694631 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.694668 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:40:29.694824 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.693104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:35019 (local address 127.8.151.193:41215)
0928 06:40:29.693443 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:29.693455 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:29.693539 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:29.693777 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:29.693780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:29.693793 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:29.694290 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:40:29.694292 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:29.694301 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:40:29.694325 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:29.694551 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:29.694554 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:29.694557 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 06:40:29.694630 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 06:40:29.694666 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":105,"threads_started":1}
I20260928 06:40:29.694898 13898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.693272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:43685 (local address 127.8.151.194:45519)
0928 06:40:29.693380 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.693384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.693680 (+ 296us) server_negotiation.cc:407] Connection header received
0928 06:40:29.693773 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.693776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.693813 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.693890 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.694549 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.694552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.694554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.694565 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.694584 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.694592 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.694664 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.694725 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.694761 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 06:40:29.695099 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.693104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:45519 (local address 127.8.151.193:43685)
0928 06:40:29.693585 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:29.693597 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:29.693686 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:29.693905 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:29.693907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:29.693919 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:29.694410 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:40:29.694412 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:29.694423 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:40:29.694446 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:29.694657 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:29.694659 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:29.694662 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 06:40:29.694729 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 06:40:29.694761 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":34,"thread_start_us":80,"threads_started":1}
I20260928 06:40:29.695533 13836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9a6cbba3dc432e917d3962b056fdb2" candidate_uuid: "5c1ae984a5574323a72999e5510b1f4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df60aae7503e4bf899583f2db4ad41cf" is_pre_election: true
I20260928 06:40:29.695596 13705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9a6cbba3dc432e917d3962b056fdb2" candidate_uuid: "5c1ae984a5574323a72999e5510b1f4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" is_pre_election: true
I20260928 06:40:29.695809 13836 raft_consensus.cc:2454] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c1ae984a5574323a72999e5510b1f4c in term 0.
I20260928 06:40:29.695834 13705 raft_consensus.cc:2454] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c1ae984a5574323a72999e5510b1f4c in term 0.
I20260928 06:40:29.696283 13508 leader_election.cc:302] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5c1ae984a5574323a72999e5510b1f4c, 8be4eec7ffe94d98a0b8060fc93dfabd, df60aae7503e4bf899583f2db4ad41cf; no voters:
I20260928 06:40:29.696527 13906 raft_consensus.cc:2781] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:29.696601 13906 raft_consensus.cc:482] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:29.696650 13906 raft_consensus.cc:3037] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:29.697628 13906 raft_consensus.cc:504] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c1ae984a5574323a72999e5510b1f4c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33749 } } peers { permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 } } peers { permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 } }
I20260928 06:40:29.697942 13906 leader_election.cc:288] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [CANDIDATE]: Term 1 election: Requested vote from peers df60aae7503e4bf899583f2db4ad41cf (127.8.151.195:35019), 8be4eec7ffe94d98a0b8060fc93dfabd (127.8.151.194:45519)
I20260928 06:40:29.698289 13705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9a6cbba3dc432e917d3962b056fdb2" candidate_uuid: "5c1ae984a5574323a72999e5510b1f4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd"
I20260928 06:40:29.698300 13836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9a6cbba3dc432e917d3962b056fdb2" candidate_uuid: "5c1ae984a5574323a72999e5510b1f4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df60aae7503e4bf899583f2db4ad41cf"
I20260928 06:40:29.698395 13705 raft_consensus.cc:3037] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:29.698405 13836 raft_consensus.cc:3037] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:29.699417 13705 raft_consensus.cc:2454] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c1ae984a5574323a72999e5510b1f4c in term 1.
I20260928 06:40:29.699465 13836 raft_consensus.cc:2454] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c1ae984a5574323a72999e5510b1f4c in term 1.
I20260928 06:40:29.699699 13510 leader_election.cc:302] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c1ae984a5574323a72999e5510b1f4c, 8be4eec7ffe94d98a0b8060fc93dfabd; no voters:
I20260928 06:40:29.699846 13906 raft_consensus.cc:2781] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:29.700048 13906 raft_consensus.cc:686] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 1 LEADER]: Becoming Leader. State: Replica: 5c1ae984a5574323a72999e5510b1f4c, State: Running, Role: LEADER
I20260928 06:40:29.700352 13906 consensus_queue.cc:237] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c1ae984a5574323a72999e5510b1f4c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33749 } } peers { permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 } } peers { permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 } }
I20260928 06:40:29.703518 13426 catalog_manager.cc:5495] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c1ae984a5574323a72999e5510b1f4c (127.8.151.193). New cstate: current_term: 1 leader_uuid: "5c1ae984a5574323a72999e5510b1f4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c1ae984a5574323a72999e5510b1f4c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 } health_report { overall_health: UNKNOWN } } }
W20260928 06:40:29.703846 13883 tablet.cc:2369] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:29.730170 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.728816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49354 (local address 127.8.151.193:33749)
0928 06:40:29.728946 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.728949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.728962 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:29.729067 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.729070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.729129 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.729192 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.729814 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.729817 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.729818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.729826 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.729840 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.729846 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.729906 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.730001 (+ 95us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.730034 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 06:40:29.736014 13836 raft_consensus.cc:1261] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf [term 1 FOLLOWER]: Refusing update from remote peer 5c1ae984a5574323a72999e5510b1f4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:29.736048 13705 raft_consensus.cc:1261] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd [term 1 FOLLOWER]: Refusing update from remote peer 5c1ae984a5574323a72999e5510b1f4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:29.736709 13906 consensus_queue.cc:1035] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8be4eec7ffe94d98a0b8060fc93dfabd" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:29.736904 13910 consensus_queue.cc:1035] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df60aae7503e4bf899583f2db4ad41cf" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:29.744023 13915 mvcc.cc:205] Tried to move back new op lower bound from 7334205971391496192 to 7334205971255246848. Current Snapshot: MvccSnapshot[applied={T|T < 7334205971391496192}]
I20260928 06:40:29.745740 13913 mvcc.cc:205] Tried to move back new op lower bound from 7334205971391496192 to 7334205971255246848. Current Snapshot: MvccSnapshot[applied={T|T < 7334205971391496192}]
I20260928 06:40:29.753520 13431 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:29.753657 13431 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:29.755924 13431 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 171faf464e5a4e5e8e2208dc5c70a3fb: Sending DeleteTablet for 3 replicas of tablet 9b9a6cbba3dc432e917d3962b056fdb2
I20260928 06:40:29.756597 13816 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9a6cbba3dc432e917d3962b056fdb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:29 UTC) from {username='slave'} at 127.0.0.1:41408
I20260928 06:40:29.756596 13685 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9a6cbba3dc432e917d3962b056fdb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:29 UTC) from {username='slave'} at 127.0.0.1:54882
I20260928 06:40:29.756618 13554 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9a6cbba3dc432e917d3962b056fdb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:29 UTC) from {username='slave'} at 127.0.0.1:49348
I20260928 06:40:29.757061 13922 tablet_replica.cc:330] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: stopping tablet replica
I20260928 06:40:29.757296 13923 tablet_replica.cc:330] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: stopping tablet replica
I20260928 06:40:29.757373 13922 raft_consensus.cc:2229] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:29.757540 13923 raft_consensus.cc:2229] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:29.757658 13922 raft_consensus.cc:2258] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:29.757774 13923 raft_consensus.cc:2258] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:29.758056 13924 tablet_replica.cc:330] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c: stopping tablet replica
I20260928 06:40:29.758143 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 13492
I20260928 06:40:29.758311 13924 raft_consensus.cc:2229] T 9b9a6cbba3dc432e917d3962b056fdb2 P 5c1ae984a5574323a72999e5510b1f4c [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:29.759157 13922 ts_tablet_manager.cc:1905] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:29.761399 13923 ts_tablet_manager.cc:1905] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:29.761804 13922 ts_tablet_manager.cc:1918] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:29.761894 13922 log.cc:1196] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/9b9a6cbba3dc432e917d3962b056fdb2
I20260928 06:40:29.762162 13922 ts_tablet_manager.cc:1939] T 9b9a6cbba3dc432e917d3962b056fdb2 P df60aae7503e4bf899583f2db4ad41cf: Deleting consensus metadata
I20260928 06:40:29.763370 13417 catalog_manager.cc:4841] TS df60aae7503e4bf899583f2db4ad41cf (127.8.151.195:35019): tablet 9b9a6cbba3dc432e917d3962b056fdb2 (table test-table [id=de1b712d6daf493eaa00238db0f677f9]) successfully deleted
I20260928 06:40:29.763859 13923 ts_tablet_manager.cc:1918] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:29.763948 13923 log.cc:1196] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/9b9a6cbba3dc432e917d3962b056fdb2
W20260928 06:40:29.763958 13416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:33749 (error 108)
I20260928 06:40:29.764216 13923 ts_tablet_manager.cc:1939] T 9b9a6cbba3dc432e917d3962b056fdb2 P 8be4eec7ffe94d98a0b8060fc93dfabd: Deleting consensus metadata
I20260928 06:40:29.764622 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 13623
I20260928 06:40:29.765283 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.764647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:33749 (local address 127.0.0.1:49364)
0928 06:40:29.765123 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:29.765201 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:33749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380}
W20260928 06:40:29.765419 13416 catalog_manager.cc:4584] TS 5c1ae984a5574323a72999e5510b1f4c (127.8.151.193:33749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9b9a6cbba3dc432e917d3962b056fdb2: Network error: Client connection negotiation failed: client connection to 127.8.151.193:33749: connect: Connection refused (error 111)
I20260928 06:40:29.770046 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 13754
I20260928 06:40:29.770782 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.770220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:45519 (local address 127.0.0.1:54884)
0928 06:40:29.770650 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:29.770698 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.194:45519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360}
I20260928 06:40:29.774346 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 13401
2026-09-28T06:40:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1061 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260928 06:40:29.794092 8799 mini_oidc.cc:152] Starting JWKS server
I20260928 06:40:29.795671 8799 webserver.cc:460] Webserver started at https://127.0.0.1:37227/ using document root <none> and password file <none>
I20260928 06:40:29.795806 8799 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 06:40:29.796025 8799 webserver.cc:460] Webserver started at http://127.0.0.1:39505/ using document root <none> and password file <none>
2026-09-28T06:40:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:29Z Disabled control of system clock
I20260928 06:40:29.804272 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:43225
--webserver_interface=127.8.151.254
--webserver_port=0
--builtin_ntp_servers=127.8.151.212:41635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:43225
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37227/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/testchainca.cert with env {}
W20260928 06:40:29.909106 13931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:29.909363 13931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:29.909401 13931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:29.913359 13931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:29.913410 13931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:29.913424 13931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:29.913439 13931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:29.913453 13931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 06:40:29.913467 13931 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 06:40:29.913479 13931 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37227/jwks.json
I20260928 06:40:29.918757 13931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:43225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:43225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37227/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:29.919799 13931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:29.920778 13931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:29.928930 13937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:29.929051 13931 server_base.cc:1034] running on GCE node
W20260928 06:40:29.928946 13936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:29.928946 13939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:29.929919 13931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:29.931010 13931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:29.932161 13931 hybrid_clock.cc:648] HybridClock initialized: now 1790577629932131 us; error 41 us; skew 500 ppm
I20260928 06:40:29.934110 13931 webserver.cc:460] Webserver started at http://127.8.151.254:43641/ using document root <none> and password file <none>
I20260928 06:40:29.934682 13931 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:29.934744 13931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:29.934954 13931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:29.937016 13931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "3348634b104b401396a34137658b0e8b"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "6c5132bf534ce8e967989dd4b8ada7d2"
server_key_iv: "40c4487f56528bf7e233d3819cf25eda"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.937565 13931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "3348634b104b401396a34137658b0e8b"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "6c5132bf534ce8e967989dd4b8ada7d2"
server_key_iv: "40c4487f56528bf7e233d3819cf25eda"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.941164 13931 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260928 06:40:29.943418 13945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:29.944521 13931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:40:29.944633 13931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "3348634b104b401396a34137658b0e8b"
format_stamp: "Formatted at 2026-09-28 06:40:29 on dist-test-slave-gtr8"
server_key: "6c5132bf534ce8e967989dd4b8ada7d2"
server_key_iv: "40c4487f56528bf7e233d3819cf25eda"
server_key_version: "encryptionkey@0"
I20260928 06:40:29.944720 13931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:29.962621 13931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:29.963292 13931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:29.963452 13931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:29.969969 13931 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:43225
I20260928 06:40:29.970167 13997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:43225 every 8 connection(s)
I20260928 06:40:29.970922 13931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:29.974756 13998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:29.979310 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 13931
I20260928 06:40:29.979427 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:29.979682 8799 external_mini_cluster.cc:1428] Setting key 467b18957966c2c34db2b7fe92878df8
I20260928 06:40:29.981618 13998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b: Bootstrap starting.
I20260928 06:40:29.983808 13998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:29.984516 13998 log.cc:824] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:29.986316 13998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b: No bootstrap required, opened a new log
I20260928 06:40:29.988705 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:29.981033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44320 (local address 127.8.151.254:43225)
0928 06:40:29.981665 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:29.981680 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:29.981715 (+ 35us) server_negotiation.cc:407] Connection header received
0928 06:40:29.982366 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:29.982382 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:29.982765 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:29.983062 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:29.983522 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:29.984498 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:29.985544 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:29.986002 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:29.986143 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:29.986880 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:29.986885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:29.986897 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:29.987310 (+ 413us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:29.987362 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:29.987383 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:29.987465 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:29.987597 (+ 132us) server_negotiation.cc:299] Negotiation successful
0928 06:40:29.987838 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":118,"threads_started":1}
I20260928 06:40:29.989398 13998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3348634b104b401396a34137658b0e8b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43225 } }
I20260928 06:40:29.989876 13998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:29.989934 13998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3348634b104b401396a34137658b0e8b, State: Initialized, Role: FOLLOWER
I20260928 06:40:29.990363 13998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3348634b104b401396a34137658b0e8b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43225 } }
I20260928 06:40:29.990573 13998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:29.990639 13998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:29.990721 13998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:29.991592 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:40:29.996451 13998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3348634b104b401396a34137658b0e8b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43225 } }
I20260928 06:40:29.996840 13998 leader_election.cc:302] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3348634b104b401396a34137658b0e8b; no voters:
I20260928 06:40:29.997200 13998 leader_election.cc:288] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:29.997308 14003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:29.997524 14003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [term 1 LEADER]: Becoming Leader. State: Replica: 3348634b104b401396a34137658b0e8b, State: Running, Role: LEADER
I20260928 06:40:29.997843 14003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3348634b104b401396a34137658b0e8b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43225 } }
I20260928 06:40:29.998513 13998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:29.999099 14004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3348634b104b401396a34137658b0e8b. Latest consensus state: current_term: 1 leader_uuid: "3348634b104b401396a34137658b0e8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3348634b104b401396a34137658b0e8b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43225 } } }
I20260928 06:40:29.999255 14004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:29.999540 14005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3348634b104b401396a34137658b0e8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3348634b104b401396a34137658b0e8b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43225 } } }
I20260928 06:40:29.999639 14005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:29.999957 14008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:30.002935 14008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:30.008234 14008 catalog_manager.cc:1347] Generated new cluster ID: f5b67dc2ceed4131925b505023db5ace
I20260928 06:40:30.008296 14008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:30.023000 14008 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:30.024255 14008 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:30.030941 14008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3348634b104b401396a34137658b0e8b: Generated new TSK 0
I20260928 06:40:30.031623 14008 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 06:40:30.064759 14028 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260928 06:40:30.064878 14028 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 06:40:30.065421 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:30.041457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.8.151.254:43225)
0928 06:40:30.041568 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:30.041572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:30.041864 (+ 292us) server_negotiation.cc:407] Connection header received
0928 06:40:30.042091 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:30.042095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:30.042149 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:30.042234 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0928 06:40:30.042495 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.043052 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.043933 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.044172 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.044259 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:30.044604 (+ 345us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 06:40:30.064945 (+ 20341us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 06:40:30.065258 (+ 313us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44326: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260928 06:40:30.065534 14001 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44326: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260928 06:40:30.067066 14027 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:30.041375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:43225 (local address 127.0.0.1:44326)
0928 06:40:30.041755 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:30.041767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:30.041918 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:30.042267 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:30.042270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:30.042279 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0928 06:40:30.042390 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:30.042395 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.043171 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.043175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:30.043818 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:30.043826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.044284 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.044288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:30.044372 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:30.044415 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 06:40:30.065215 (+ 20800us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 06:40:30.065508 (+ 293us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.151.254:43225: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":107,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.151.254:43225: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260928 06:40:30.067713 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 13931
2026-09-28T06:40:30Z chronyd exiting
I20260928 06:40:30.104142 8799 test_util.cc:175] -----------------------------------------------
I20260928 06:40:30.104255 8799 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790577605970741-8799-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (311 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260928 06:40:30.105238 8799 mini_oidc.cc:152] Starting JWKS server
I20260928 06:40:30.106045 8799 webserver.cc:460] Webserver started at https://127.0.0.1:45123/ using document root <none> and password file <none>
I20260928 06:40:30.106134 8799 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 06:40:30.106364 8799 webserver.cc:460] Webserver started at http://127.0.0.1:33503/ using document root <none> and password file <none>
2026-09-28T06:40:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:30Z Disabled control of system clock
I20260928 06:40:30.114162 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:44963
--webserver_interface=127.8.151.254
--webserver_port=0
--builtin_ntp_servers=127.8.151.212:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:44963
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45123/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/testchainca.cert with env {}
W20260928 06:40:30.218642 14034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:30.218899 14034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:30.218936 14034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:30.222813 14034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:30.222867 14034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:30.222882 14034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:30.222896 14034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:30.222909 14034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 06:40:30.222922 14034 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 06:40:30.222939 14034 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45123/jwks.json
I20260928 06:40:30.228246 14034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:44963
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:44963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45123/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:30.229436 14034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:30.230454 14034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:30.238795 14039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:30.238828 14040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:30.238795 14042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:30.239076 14034 server_base.cc:1034] running on GCE node
I20260928 06:40:30.239756 14034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:30.240859 14034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:30.242027 14034 hybrid_clock.cc:648] HybridClock initialized: now 1790577630242008 us; error 29 us; skew 500 ppm
I20260928 06:40:30.243873 14034 webserver.cc:460] Webserver started at http://127.8.151.254:34973/ using document root <none> and password file <none>
I20260928 06:40:30.244441 14034 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:30.244506 14034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:30.244695 14034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:30.246681 14034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "a592f49848b040228ab22e67e8224438"
format_stamp: "Formatted at 2026-09-28 06:40:30 on dist-test-slave-gtr8"
server_key: "b93e0dd7558ee389e865d85a352617c7"
server_key_iv: "21cbc4927322117dc56f67bcd28093b6"
server_key_version: "encryptionkey@0"
I20260928 06:40:30.247246 14034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "a592f49848b040228ab22e67e8224438"
format_stamp: "Formatted at 2026-09-28 06:40:30 on dist-test-slave-gtr8"
server_key: "b93e0dd7558ee389e865d85a352617c7"
server_key_iv: "21cbc4927322117dc56f67bcd28093b6"
server_key_version: "encryptionkey@0"
I20260928 06:40:30.250748 14034 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:40:30.253142 14048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:30.254422 14034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:30.254546 14034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "a592f49848b040228ab22e67e8224438"
format_stamp: "Formatted at 2026-09-28 06:40:30 on dist-test-slave-gtr8"
server_key: "b93e0dd7558ee389e865d85a352617c7"
server_key_iv: "21cbc4927322117dc56f67bcd28093b6"
server_key_version: "encryptionkey@0"
I20260928 06:40:30.254670 14034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:30.264277 14034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:30.264892 14034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:30.265050 14034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:30.271608 14034 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:44963
I20260928 06:40:30.271623 14100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:44963 every 8 connection(s)
I20260928 06:40:30.272529 14034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:30.275669 14101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:30.278949 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 14034
I20260928 06:40:30.279078 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:30.279395 8799 external_mini_cluster.cc:1428] Setting key 931427fd7fa4c9a3c24ff2701f0c3ded
I20260928 06:40:30.281700 14101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438: Bootstrap starting.
I20260928 06:40:30.283957 14101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:30.284796 14101 log.cc:824] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:30.287093 14101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438: No bootstrap required, opened a new log
I20260928 06:40:30.288460 14104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:30.280715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.8.151.254:44963)
0928 06:40:30.281305 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:30.281320 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:30.281354 (+ 34us) server_negotiation.cc:407] Connection header received
0928 06:40:30.282025 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:30.282044 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:30.282432 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:30.282704 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:30.283045 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.284075 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.285241 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.285838 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.285952 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:30.286767 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:30.286772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:30.286784 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:30.287196 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:30.287281 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:30.287304 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:30.287380 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:30.287519 (+ 139us) server_negotiation.cc:299] Negotiation successful
0928 06:40:30.287782 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":109,"threads_started":1}
I20260928 06:40:30.290767 14101 raft_consensus.cc:348] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592f49848b040228ab22e67e8224438" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44963 } }
I20260928 06:40:30.290984 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:40:30.291018 14101 raft_consensus.cc:374] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:30.291064 14101 raft_consensus.cc:729] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a592f49848b040228ab22e67e8224438, State: Initialized, Role: FOLLOWER
I20260928 06:40:30.291507 14101 consensus_queue.cc:260] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592f49848b040228ab22e67e8224438" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44963 } }
I20260928 06:40:30.291639 14101 raft_consensus.cc:388] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:30.291692 14101 raft_consensus.cc:482] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:30.291777 14101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:30.292789 14101 raft_consensus.cc:504] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592f49848b040228ab22e67e8224438" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44963 } }
I20260928 06:40:30.293131 14101 leader_election.cc:302] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a592f49848b040228ab22e67e8224438; no voters:
I20260928 06:40:30.293435 14101 leader_election.cc:288] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:30.293519 14106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:30.293740 14106 raft_consensus.cc:686] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [term 1 LEADER]: Becoming Leader. State: Replica: a592f49848b040228ab22e67e8224438, State: Running, Role: LEADER
I20260928 06:40:30.294062 14106 consensus_queue.cc:237] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592f49848b040228ab22e67e8224438" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44963 } }
I20260928 06:40:30.294652 14101 sys_catalog.cc:564] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:30.295296 14106 sys_catalog.cc:455] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a592f49848b040228ab22e67e8224438. Latest consensus state: current_term: 1 leader_uuid: "a592f49848b040228ab22e67e8224438" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592f49848b040228ab22e67e8224438" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44963 } } }
I20260928 06:40:30.295444 14106 sys_catalog.cc:458] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:30.295296 14108 sys_catalog.cc:455] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a592f49848b040228ab22e67e8224438" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592f49848b040228ab22e67e8224438" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44963 } } }
I20260928 06:40:30.295533 14108 sys_catalog.cc:458] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:30.295933 14113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:30.298916 14113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:30.304240 14113 catalog_manager.cc:1347] Generated new cluster ID: 98dd13f0f57041de96dfc4f834b4129f
I20260928 06:40:30.304304 14113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:30.315119 14113 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:30.316045 14113 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:30.325306 14113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a592f49848b040228ab22e67e8224438: Generated new TSK 0
I20260928 06:40:30.325930 14113 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 06:40:30.360512 14131 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260928 06:40:30.360612 14131 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 06:40:30.361155 14104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:30.338042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56838 (local address 127.8.151.254:44963)
0928 06:40:30.338175 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:30.338179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:30.338371 (+ 192us) server_negotiation.cc:407] Connection header received
0928 06:40:30.338506 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:30.338509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:30.338554 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:30.338637 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0928 06:40:30.338864 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.339565 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.340809 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.341021 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.341094 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:30.341374 (+ 280us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 06:40:30.360824 (+ 19450us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 06:40:30.361042 (+ 218us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56838: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":51}
W20260928 06:40:30.361213 14104 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56838: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260928 06:40:30.361371 14130 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:30.337952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:44963 (local address 127.0.0.1:56838)
0928 06:40:30.338270 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:30.338284 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:30.338395 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:30.338649 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:30.338652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:30.338660 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0928 06:40:30.338769 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:30.338775 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.339676 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.339681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:30.340689 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:30.340700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.341117 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.341121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:30.341216 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:30.341243 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 06:40:30.361007 (+ 19764us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 06:40:30.361196 (+ 189us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.151.254:44963: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":98,"threads_started":1}
I20260928 06:40:30.361958 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 14034
2026-09-28T06:40:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (296 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260928 06:40:30.401684 8799 mini_oidc.cc:152] Starting JWKS server
I20260928 06:40:30.402524 8799 webserver.cc:460] Webserver started at https://127.0.0.1:37289/ using document root <none> and password file <none>
I20260928 06:40:30.402608 8799 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 06:40:30.402866 8799 webserver.cc:460] Webserver started at http://127.0.0.1:43369/ using document root <none> and password file <none>
2026-09-28T06:40:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:30Z Disabled control of system clock
I20260928 06:40:30.411382 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:34021
--webserver_interface=127.8.151.254
--webserver_port=0
--builtin_ntp_servers=127.8.151.212:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:34021
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37289/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 06:40:30.522595 14137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:30.522845 14137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:30.522882 14137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:30.526821 14137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:30.526870 14137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:30.526883 14137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:30.526897 14137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:30.526911 14137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 06:40:30.526924 14137 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 06:40:30.526942 14137 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37289/jwks.json
I20260928 06:40:30.531558 14137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:34021
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:34021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37289/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:30.532521 14137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:30.533792 14137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:30.542111 14142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:30.542145 14145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:30.542101 14143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:30.542366 14137 server_base.cc:1034] running on GCE node
I20260928 06:40:30.542975 14137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:30.544093 14137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:30.545282 14137 hybrid_clock.cc:648] HybridClock initialized: now 1790577630545266 us; error 48 us; skew 500 ppm
I20260928 06:40:30.551096 14137 webserver.cc:460] Webserver started at http://127.8.151.254:35383/ using document root <none> and password file <none>
I20260928 06:40:30.551674 14137 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:30.551743 14137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:30.551937 14137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:30.553936 14137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "65284e8bbd70476f922e21079d5c2be9"
format_stamp: "Formatted at 2026-09-28 06:40:30 on dist-test-slave-gtr8"
server_key: "dd7cfca941e12ca7873b983cd3ded95d"
server_key_iv: "cdac4bd3066dab6a4a12c020cccb2f15"
server_key_version: "encryptionkey@0"
I20260928 06:40:30.554469 14137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "65284e8bbd70476f922e21079d5c2be9"
format_stamp: "Formatted at 2026-09-28 06:40:30 on dist-test-slave-gtr8"
server_key: "dd7cfca941e12ca7873b983cd3ded95d"
server_key_iv: "cdac4bd3066dab6a4a12c020cccb2f15"
server_key_version: "encryptionkey@0"
I20260928 06:40:30.558225 14137 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260928 06:40:30.560639 14151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:30.561878 14137 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:30.562000 14137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "65284e8bbd70476f922e21079d5c2be9"
format_stamp: "Formatted at 2026-09-28 06:40:30 on dist-test-slave-gtr8"
server_key: "dd7cfca941e12ca7873b983cd3ded95d"
server_key_iv: "cdac4bd3066dab6a4a12c020cccb2f15"
server_key_version: "encryptionkey@0"
I20260928 06:40:30.562101 14137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:30.573342 14137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:30.573951 14137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:30.574107 14137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:30.580597 14137 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:34021
I20260928 06:40:30.580608 14203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:34021 every 8 connection(s)
I20260928 06:40:30.581468 14137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:30.584156 14204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:30.586508 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 14137
I20260928 06:40:30.586629 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:30.586938 8799 external_mini_cluster.cc:1428] Setting key f756d6836bcb068dad11b216f9f4f377
I20260928 06:40:30.590390 14204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9: Bootstrap starting.
I20260928 06:40:30.592502 14204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:30.593209 14204 log.cc:824] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:30.595669 14207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:30.588303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33740 (local address 127.8.151.254:34021)
0928 06:40:30.588912 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:30.588927 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:30.588960 (+ 33us) server_negotiation.cc:407] Connection header received
0928 06:40:30.589583 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:30.589598 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:30.589978 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:30.590269 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:30.590679 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.591688 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.592535 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.593084 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.593284 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:30.594019 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:30.594026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:30.594039 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:30.594402 (+ 363us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:30.594457 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:30.594477 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:30.594559 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:30.594683 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 06:40:30.594944 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260928 06:40:30.596421 14204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9: No bootstrap required, opened a new log
I20260928 06:40:30.598461 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:40:30.600220 14204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65284e8bbd70476f922e21079d5c2be9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34021 } }
I20260928 06:40:30.600464 14204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:30.600518 14204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65284e8bbd70476f922e21079d5c2be9, State: Initialized, Role: FOLLOWER
I20260928 06:40:30.600955 14204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65284e8bbd70476f922e21079d5c2be9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34021 } }
I20260928 06:40:30.601082 14204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:30.601141 14204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:30.601234 14204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:30.688334 14204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65284e8bbd70476f922e21079d5c2be9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34021 } }
I20260928 06:40:30.688923 14204 leader_election.cc:302] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65284e8bbd70476f922e21079d5c2be9; no voters:
I20260928 06:40:30.689386 14204 leader_election.cc:288] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:30.689640 14209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:30.689924 14209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [term 1 LEADER]: Becoming Leader. State: Replica: 65284e8bbd70476f922e21079d5c2be9, State: Running, Role: LEADER
I20260928 06:40:30.690797 14209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65284e8bbd70476f922e21079d5c2be9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34021 } }
I20260928 06:40:30.690872 14204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 06:40:30.696058 14223 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:40:30.696130 14223 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:40:30.790566 14210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65284e8bbd70476f922e21079d5c2be9. Latest consensus state: current_term: 1 leader_uuid: "65284e8bbd70476f922e21079d5c2be9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65284e8bbd70476f922e21079d5c2be9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34021 } } }
I20260928 06:40:30.790819 14210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:30.790999 14211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65284e8bbd70476f922e21079d5c2be9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65284e8bbd70476f922e21079d5c2be9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34021 } } }
I20260928 06:40:30.791249 14211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:30.791504 14226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:30.792606 14226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:30.798843 14226 catalog_manager.cc:1347] Generated new cluster ID: 6bf964a424f341e5964daeee76e292b9
I20260928 06:40:30.798916 14226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:30.806097 14226 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:30.806948 14226 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:30.816367 14226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65284e8bbd70476f922e21079d5c2be9: Generated new TSK 0
I20260928 06:40:30.817036 14226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:40:30.848673 14207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:30.843661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33750 (local address 127.8.151.254:34021)
0928 06:40:30.843786 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:30.843791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:30.843916 (+ 125us) server_negotiation.cc:407] Connection header received
0928 06:40:30.844074 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:30.844076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:30.844121 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:30.844202 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0928 06:40:30.844447 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.844961 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.846163 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.846377 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.846453 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:30.846747 (+ 294us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 06:40:30.848320 (+ 1573us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37289/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 06:40:30.848528 (+ 208us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33750: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37289/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":52}
W20260928 06:40:30.848742 14207 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33750: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37289/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260928 06:40:30.850401 14233 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:30.843547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:34021 (local address 127.0.0.1:33750)
0928 06:40:30.843843 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:30.843857 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:30.843968 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:30.844229 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:30.844232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:30.844240 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0928 06:40:30.844360 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:30.844366 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.845079 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.845083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:30.846060 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:30.846071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:30.846490 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:30.846495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:30.846582 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:30.846610 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 06:40:30.848480 (+ 1870us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37289/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 06:40:30.850222 (+ 1742us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.151.254:34021: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37289/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":1538,"thread_start_us":96,"threads_started":1}
I20260928 06:40:30.851322 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 14137
E20260928 06:40:30.861629 14234 webserver.cc:533] Webserver: sslize failed:
E20260928 06:40:30.861709 14234 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-28T06:40:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (490 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-28T06:40:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:30Z Disabled control of system clock
I20260928 06:40:30.901010 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:46325
--webserver_interface=127.8.151.254
--webserver_port=0
--builtin_ntp_servers=127.8.151.212:33787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:46325
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/testchainca.cert with env {}
W20260928 06:40:31.010047 14238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:31.010331 14238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:31.010370 14238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:31.014544 14238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:31.014613 14238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:31.014633 14238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:31.014649 14238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:31.014663 14238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 06:40:31.014683 14238 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 06:40:31.014709 14238 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260928 06:40:31.020303 14238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:33787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:46325
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:46325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:31.021627 14238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:31.022687 14238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:31.030818 14246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:31.030843 14244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:31.030841 14243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:31.031221 14238 server_base.cc:1034] running on GCE node
I20260928 06:40:31.031762 14238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:31.032806 14238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:31.033986 14238 hybrid_clock.cc:648] HybridClock initialized: now 1790577631033970 us; error 28 us; skew 500 ppm
I20260928 06:40:31.035885 14238 webserver.cc:460] Webserver started at http://127.8.151.254:44587/ using document root <none> and password file <none>
I20260928 06:40:31.036456 14238 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:31.036525 14238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:31.036707 14238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:31.038728 14238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "89daca1262ed4f7f8feac60c901f9e6e"
format_stamp: "Formatted at 2026-09-28 06:40:31 on dist-test-slave-gtr8"
server_key: "74415f5be6dc5c38c64b82c8e796fe0a"
server_key_iv: "0d6ca530a041c5936d084e992fa9c230"
server_key_version: "encryptionkey@0"
I20260928 06:40:31.039309 14238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "89daca1262ed4f7f8feac60c901f9e6e"
format_stamp: "Formatted at 2026-09-28 06:40:31 on dist-test-slave-gtr8"
server_key: "74415f5be6dc5c38c64b82c8e796fe0a"
server_key_iv: "0d6ca530a041c5936d084e992fa9c230"
server_key_version: "encryptionkey@0"
I20260928 06:40:31.043002 14238 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:31.045377 14252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:31.046645 14238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:31.046757 14238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "89daca1262ed4f7f8feac60c901f9e6e"
format_stamp: "Formatted at 2026-09-28 06:40:31 on dist-test-slave-gtr8"
server_key: "74415f5be6dc5c38c64b82c8e796fe0a"
server_key_iv: "0d6ca530a041c5936d084e992fa9c230"
server_key_version: "encryptionkey@0"
I20260928 06:40:31.046859 14238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:31.064543 14238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:31.065303 14238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:31.065477 14238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:31.072103 14238 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:46325
I20260928 06:40:31.072199 14304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:46325 every 8 connection(s)
I20260928 06:40:31.073045 14238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:31.076221 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 14238
I20260928 06:40:31.076339 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:31.076406 14305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:31.076659 8799 external_mini_cluster.cc:1428] Setting key 5e6b7571ccf67612ec61a8e2cdbcd420
I20260928 06:40:31.084244 14305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e: Bootstrap starting.
I20260928 06:40:31.086381 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:31.078077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.8.151.254:46325)
0928 06:40:31.078719 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:31.078735 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:31.078773 (+ 38us) server_negotiation.cc:407] Connection header received
0928 06:40:31.079494 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:31.079511 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:31.079918 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:31.080208 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:31.080553 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:31.081514 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:31.082704 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:31.083274 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:31.083404 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:31.084264 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:31.084269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:31.084281 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:40:31.084716 (+ 435us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:31.084774 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:40:31.084796 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:31.084878 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:31.085085 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 06:40:31.085360 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":120,"threads_started":1}
I20260928 06:40:31.086704 14305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:31.087550 14305 log.cc:824] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:31.089290 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:40:31.089916 14305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e: No bootstrap required, opened a new log
I20260928 06:40:31.093425 14305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89daca1262ed4f7f8feac60c901f9e6e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46325 } }
I20260928 06:40:31.093663 14305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:31.093698 14305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89daca1262ed4f7f8feac60c901f9e6e, State: Initialized, Role: FOLLOWER
I20260928 06:40:31.094091 14305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89daca1262ed4f7f8feac60c901f9e6e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46325 } }
I20260928 06:40:31.094213 14305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:31.094259 14305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:31.094355 14305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:31.187453 14305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89daca1262ed4f7f8feac60c901f9e6e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46325 } }
I20260928 06:40:31.188019 14305 leader_election.cc:302] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89daca1262ed4f7f8feac60c901f9e6e; no voters:
I20260928 06:40:31.188539 14305 leader_election.cc:288] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:31.188699 14310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:31.188938 14310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [term 1 LEADER]: Becoming Leader. State: Replica: 89daca1262ed4f7f8feac60c901f9e6e, State: Running, Role: LEADER
I20260928 06:40:31.189294 14310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89daca1262ed4f7f8feac60c901f9e6e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46325 } }
I20260928 06:40:31.189982 14305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:31.191090 14311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89daca1262ed4f7f8feac60c901f9e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89daca1262ed4f7f8feac60c901f9e6e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46325 } } }
I20260928 06:40:31.191237 14311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:31.191056 14312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89daca1262ed4f7f8feac60c901f9e6e. Latest consensus state: current_term: 1 leader_uuid: "89daca1262ed4f7f8feac60c901f9e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89daca1262ed4f7f8feac60c901f9e6e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46325 } } }
I20260928 06:40:31.191596 14312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:31.191884 14319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:31.195796 14319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:31.201313 14319 catalog_manager.cc:1347] Generated new cluster ID: d718927baf4047fda1674b17c95dc449
I20260928 06:40:31.201390 14319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:31.243659 14319 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:31.244917 14319 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:31.260210 14319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89daca1262ed4f7f8feac60c901f9e6e: Generated new TSK 0
I20260928 06:40:31.260953 14319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:40:31.423369 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:31.285212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38144 (local address 127.8.151.254:46325)
0928 06:40:31.285357 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:31.285362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:31.285525 (+ 163us) server_negotiation.cc:407] Connection header received
0928 06:40:31.285691 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:31.285693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:31.285739 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:31.285830 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0928 06:40:31.286055 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:31.286550 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:31.287780 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:31.287997 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:31.288067 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:31.288363 (+ 296us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 06:40:31.422901 (+134538us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 06:40:31.423233 (+ 332us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38144: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":49}
W20260928 06:40:31.423454 14308 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38144: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260928 06:40:31.423604 14334 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:31.285119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:46325 (local address 127.0.0.1:38144)
0928 06:40:31.285441 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:31.285455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:31.285569 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:31.285845 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:31.285848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:31.285855 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0928 06:40:31.285970 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:31.285977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:31.286653 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:31.286657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:31.287670 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:31.287681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:31.288096 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:31.288100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:31.288186 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:31.288216 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 06:40:31.423209 (+134993us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 06:40:31.423409 (+ 200us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.151.254:46325: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":98,"threads_started":1}
I20260928 06:40:31.424108 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 14238
2026-09-28T06:40:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (550 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (122 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (120 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:31 dist-test-slave-gtr8 krb5kdc[14346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:31 dist-test-slave-gtr8 krb5kdc[14346](info): set up 2 sockets
Sep 28 06:40:31 dist-test-slave-gtr8 krb5kdc[14346](info): commencing operation
krb5kdc: starting...
W20260928 06:40:32.711647 8799 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:32 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:32.830437 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:40263
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:40263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:32.935772 14359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:32.936033 14359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:32.936088 14359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:32.940035 14359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:32.940088 14359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:32.940114 14359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:32.940136 14359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:32.940163 14359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:32.945328 14359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:40263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:40263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:32.946307 14359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:32.947347 14359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:32.955644 14364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:32.955669 14367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:32.955744 14365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:32.955801 14359 server_base.cc:1034] running on GCE node
I20260928 06:40:32.956393 14359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:32.957461 14359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:32.958642 14359 hybrid_clock.cc:648] HybridClock initialized: now 1790577632958615 us; error 30 us; skew 500 ppm
Sep 28 06:40:32 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577632, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:32.961562 14359 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:32.962416 14359 webserver.cc:460] Webserver started at http://127.8.151.254:40605/ using document root <none> and password file <none>
I20260928 06:40:32.962937 14359 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:32.962999 14359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:32.963217 14359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:32.965184 14359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "cc5a56d5ec4646d4927d340e7dfff972"
format_stamp: "Formatted at 2026-09-28 06:40:32 on dist-test-slave-gtr8"
server_key: "bd8948b06f8223846bf70e41740c2b17"
server_key_iv: "ca43a3b214739d757f1f1e85fa4791c5"
server_key_version: "encryptionkey@0"
I20260928 06:40:32.965719 14359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "cc5a56d5ec4646d4927d340e7dfff972"
format_stamp: "Formatted at 2026-09-28 06:40:32 on dist-test-slave-gtr8"
server_key: "bd8948b06f8223846bf70e41740c2b17"
server_key_iv: "ca43a3b214739d757f1f1e85fa4791c5"
server_key_version: "encryptionkey@0"
I20260928 06:40:32.969362 14359 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:32.971650 14374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:32.972914 14359 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:32.973037 14359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "cc5a56d5ec4646d4927d340e7dfff972"
format_stamp: "Formatted at 2026-09-28 06:40:32 on dist-test-slave-gtr8"
server_key: "bd8948b06f8223846bf70e41740c2b17"
server_key_iv: "ca43a3b214739d757f1f1e85fa4791c5"
server_key_version: "encryptionkey@0"
I20260928 06:40:32.973129 14359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:33.006598 14359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:33.009657 14359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:33.009837 14359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:33.016494 14359 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:40263
I20260928 06:40:33.016508 14426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:40263 every 8 connection(s)
I20260928 06:40:33.017418 14359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:33.020217 14427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:33.026052 14427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972: Bootstrap starting.
I20260928 06:40:33.026379 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 14359
I20260928 06:40:33.026492 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:33.026721 8799 external_mini_cluster.cc:1428] Setting key 97a3629a45a809ae41dd246b5e26013d
I20260928 06:40:33.028673 14427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:33.029567 14427 log.cc:824] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:33.031814 14427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972: No bootstrap required, opened a new log
I20260928 06:40:33.035717 14427 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5a56d5ec4646d4927d340e7dfff972" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40263 } }
I20260928 06:40:33.036013 14427 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:33.036069 14427 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc5a56d5ec4646d4927d340e7dfff972, State: Initialized, Role: FOLLOWER
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:33.036499 14427 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5a56d5ec4646d4927d340e7dfff972" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40263 } }
I20260928 06:40:33.036646 14427 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:33.036715 14427 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:33.036814 14427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:33.037897 14427 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5a56d5ec4646d4927d340e7dfff972" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40263 } }
I20260928 06:40:33.038276 14427 leader_election.cc:302] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc5a56d5ec4646d4927d340e7dfff972; no voters:
I20260928 06:40:33.038661 14427 leader_election.cc:288] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:33.038743 14432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:33.038964 14432 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [term 1 LEADER]: Becoming Leader. State: Replica: cc5a56d5ec4646d4927d340e7dfff972, State: Running, Role: LEADER
I20260928 06:40:33.039422 14432 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5a56d5ec4646d4927d340e7dfff972" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40263 } }
I20260928 06:40:33.039974 14427 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:33.041288 14433 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc5a56d5ec4646d4927d340e7dfff972" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5a56d5ec4646d4927d340e7dfff972" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40263 } } }
I20260928 06:40:33.041412 14433 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:33.041795 14430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:33.028090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.8.151.254:40263)
0928 06:40:33.028537 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:33.028555 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:33.028598 (+ 43us) server_negotiation.cc:407] Connection header received
0928 06:40:33.029460 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:33.029483 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:33.029922 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:33.030217 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:33.030957 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.032373 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.033254 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.033958 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.034113 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:33.036644 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:33.036650 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:33.036667 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:33.036699 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:33.038531 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.039049 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.039056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.039062 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.039242 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.039608 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.039612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.039614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.040129 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:33.040327 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:33.040618 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 06:40:33.040917 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":126,"threads_started":1}
I20260928 06:40:33.043313 14434 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc5a56d5ec4646d4927d340e7dfff972. Latest consensus state: current_term: 1 leader_uuid: "cc5a56d5ec4646d4927d340e7dfff972" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5a56d5ec4646d4927d340e7dfff972" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40263 } } }
I20260928 06:40:33.043671 14434 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972 [sys.catalog]: This master's current role is: LEADER
W20260928 06:40:33.044915 14447 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:40:33.044977 14447 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:40:33.045068 14442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:33.045745 14442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:33.051169 14442 catalog_manager.cc:1347] Generated new cluster ID: e99bf85d4d9d441bb7ec72c5452f24e2
I20260928 06:40:33.051234 14442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:33.086977 14442 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:33.088297 14442 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:33.093596 14442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc5a56d5ec4646d4927d340e7dfff972: Generated new TSK 0
I20260928 06:40:33.094398 14442 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:40:33.126150 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40263
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:33.234850 14453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:33.235129 14453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:33.235203 14453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:33.239214 14453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:33.239275 14453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:33.239338 14453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:33.244707 14453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:33.245779 14453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:33.246783 14453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:33.256024 14458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:33.256024 14459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:33.256095 14453 server_base.cc:1034] running on GCE node
W20260928 06:40:33.256027 14461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:33.256585 14453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:33.257202 14453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:33.258371 14453 hybrid_clock.cc:648] HybridClock initialized: now 1790577633258352 us; error 32 us; skew 500 ppm
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:33.260999 14453 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:40:33.261989 14453 webserver.cc:460] Webserver started at http://127.8.151.193:33155/ using document root <none> and password file <none>
I20260928 06:40:33.262487 14453 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:33.262540 14453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:33.262696 14453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:33.264648 14453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "6d8c7b9dc1ee4e7da6ddb13078dc5311"
format_stamp: "Formatted at 2026-09-28 06:40:33 on dist-test-slave-gtr8"
server_key: "e49b42056fadacceb62541c04d32e4d9"
server_key_iv: "468d743e42416be630a41f635e17716e"
server_key_version: "encryptionkey@0"
I20260928 06:40:33.265141 14453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "6d8c7b9dc1ee4e7da6ddb13078dc5311"
format_stamp: "Formatted at 2026-09-28 06:40:33 on dist-test-slave-gtr8"
server_key: "e49b42056fadacceb62541c04d32e4d9"
server_key_iv: "468d743e42416be630a41f635e17716e"
server_key_version: "encryptionkey@0"
I20260928 06:40:33.268608 14453 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:33.270838 14468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:33.271996 14453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:33.272085 14453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "6d8c7b9dc1ee4e7da6ddb13078dc5311"
format_stamp: "Formatted at 2026-09-28 06:40:33 on dist-test-slave-gtr8"
server_key: "e49b42056fadacceb62541c04d32e4d9"
server_key_iv: "468d743e42416be630a41f635e17716e"
server_key_version: "encryptionkey@0"
I20260928 06:40:33.272166 14453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:33.294004 14453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:33.301975 14453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:33.302130 14453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:33.302685 14453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:33.303526 14453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:33.303573 14453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:33.303610 14453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:33.303624 14453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:33.311918 14453 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:33071
I20260928 06:40:33.311957 14581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:33071 every 8 connection(s)
I20260928 06:40:33.312794 14453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:33.322365 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 14453
I20260928 06:40:33.322466 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:33.322707 8799 external_mini_cluster.cc:1428] Setting key ceb1682f458786e49c0f6bea6718cef3
I20260928 06:40:33.325739 14430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:33.314534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:34255 (local address 127.8.151.254:40263)
0928 06:40:33.314678 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:33.314683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:33.315538 (+ 855us) server_negotiation.cc:407] Connection header received
0928 06:40:33.316858 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:33.316861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:33.316912 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:33.316986 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:33.318093 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.318662 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.319779 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.320014 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.320094 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:33.322246 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:33.322249 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:33.322251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:33.322275 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:33.323726 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.324239 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.324241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.324243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.324317 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.324678 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.324680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.324682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.324856 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:33.324928 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:33.325477 (+ 549us) server_negotiation.cc:299] Negotiation successful
0928 06:40:33.325577 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:40:33.326576 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:33.314925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:40263 (local address 127.8.151.193:34255)
0928 06:40:33.315396 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:33.315439 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:33.316641 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:33.317130 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:33.317136 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:33.317466 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:33.317943 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:33.317957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.318756 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.318758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:33.319665 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:33.319673 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.320129 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.320132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:33.320263 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:33.320736 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:33.320751 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:33.322088 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:33.323829 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:33.323841 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:33.323855 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:33.324145 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:33.324400 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:33.324403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:33.324404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:33.324576 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:33.324944 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:33.324951 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:33.325235 (+ 284us) client_negotiation.cc:769] Sending connection context
0928 06:40:33.325425 (+ 190us) client_negotiation.cc:240] Negotiation successful
0928 06:40:33.325748 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":100,"threads_started":1}
I20260928 06:40:33.327699 14582 heartbeater.cc:347] Connected to a master server at 127.8.151.254:40263
I20260928 06:40:33.327955 14582 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:33.328531 14582 heartbeater.cc:511] Master 127.8.151.254:40263 requested a full tablet report, sending...
I20260928 06:40:33.330037 14391 ts_manager.cc:194] Registered new tserver with Master: 6d8c7b9dc1ee4e7da6ddb13078dc5311 (127.8.151.193:33071)
I20260928 06:40:33.331131 14391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:34255
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:40:33.348435 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40263
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:33.460320 14587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:33.460565 14587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:33.460623 14587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:33.466303 14587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:33.466382 14587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:33.466472 14587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:33.474193 14587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:33.475627 14587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:33.476976 14587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:33.486693 14593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:33.486750 14592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:33.486734 14595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:33.486948 14587 server_base.cc:1034] running on GCE node
I20260928 06:40:33.487478 14587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:33.488133 14587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:33.489300 14587 hybrid_clock.cc:648] HybridClock initialized: now 1790577633489250 us; error 55 us; skew 500 ppm
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:33.491850 14587 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:40:33.492704 14587 webserver.cc:460] Webserver started at http://127.8.151.194:44403/ using document root <none> and password file <none>
I20260928 06:40:33.493279 14587 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:33.493366 14587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:33.493573 14587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:33.496143 14587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "581f0b3289774e449ff31cb6c3aed009"
format_stamp: "Formatted at 2026-09-28 06:40:33 on dist-test-slave-gtr8"
server_key: "74d93904ff340ab779b5ed4db8b714b4"
server_key_iv: "ea750e15935e9f602dc8fced6558c4fb"
server_key_version: "encryptionkey@0"
I20260928 06:40:33.496754 14587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "581f0b3289774e449ff31cb6c3aed009"
format_stamp: "Formatted at 2026-09-28 06:40:33 on dist-test-slave-gtr8"
server_key: "74d93904ff340ab779b5ed4db8b714b4"
server_key_iv: "ea750e15935e9f602dc8fced6558c4fb"
server_key_version: "encryptionkey@0"
I20260928 06:40:33.500779 14587 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260928 06:40:33.502969 14602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:33.504175 14587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:33.504285 14587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "581f0b3289774e449ff31cb6c3aed009"
format_stamp: "Formatted at 2026-09-28 06:40:33 on dist-test-slave-gtr8"
server_key: "74d93904ff340ab779b5ed4db8b714b4"
server_key_iv: "ea750e15935e9f602dc8fced6558c4fb"
server_key_version: "encryptionkey@0"
I20260928 06:40:33.504405 14587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:33.524510 14587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:33.527108 14587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:33.527303 14587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:33.527877 14587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:33.528644 14587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:33.528689 14587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:33.528739 14587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:33.528766 14587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:33.536976 14587 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:42155
I20260928 06:40:33.537001 14715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:42155 every 8 connection(s)
I20260928 06:40:33.537815 14587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:33.544001 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 14587
I20260928 06:40:33.544103 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:33.544350 8799 external_mini_cluster.cc:1428] Setting key 5ef3132ed51e209d539fc767929d3e9e
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:33.551047 14430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:33.539505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:58255 (local address 127.8.151.254:40263)
0928 06:40:33.539647 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:33.539651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:33.540488 (+ 837us) server_negotiation.cc:407] Connection header received
0928 06:40:33.542048 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:33.542051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:33.542099 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:33.542169 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:33.543305 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.543884 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.545274 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.545556 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.545662 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:33.547960 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:33.547964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:33.547966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:33.547992 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:33.549212 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.549683 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.549687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.549689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.549768 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.550054 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.550057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.550059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.550241 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:33.550324 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:33.550774 (+ 450us) server_negotiation.cc:299] Negotiation successful
0928 06:40:33.550889 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 06:40:33.551808 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:33.539901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:40263 (local address 127.8.151.194:58255)
0928 06:40:33.540315 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:33.540362 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:33.541860 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:33.542361 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:33.542368 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:33.542694 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:33.543175 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:33.543191 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.543999 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.544001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:33.545133 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:33.545144 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.545711 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.545715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:33.545898 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:33.546423 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:33.546438 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:33.547811 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:33.549346 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:33.549351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:33.549364 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:33.549594 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:33.549844 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:33.549847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:33.549850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:33.549970 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:33.550333 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:33.550339 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:33.550558 (+ 219us) client_negotiation.cc:769] Sending connection context
0928 06:40:33.550717 (+ 159us) client_negotiation.cc:240] Negotiation successful
0928 06:40:33.551017 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
I20260928 06:40:33.553014 14716 heartbeater.cc:347] Connected to a master server at 127.8.151.254:40263
I20260928 06:40:33.553222 14716 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:33.553673 14716 heartbeater.cc:511] Master 127.8.151.254:40263 requested a full tablet report, sending...
I20260928 06:40:33.554606 14391 ts_manager.cc:194] Registered new tserver with Master: 581f0b3289774e449ff31cb6c3aed009 (127.8.151.194:42155)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
I20260928 06:40:33.555326 14391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:58255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:40:33.569928 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40263
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:33.675684 14721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:33.675940 14721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:33.675981 14721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:33.680048 14721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:33.680109 14721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:33.680169 14721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:33.685352 14721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:33.686342 14721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:33.687350 14721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:33.696408 14729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:33.696416 14726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:33.696439 14727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:33.696650 14721 server_base.cc:1034] running on GCE node
I20260928 06:40:33.697033 14721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:33.697626 14721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:33.698796 14721 hybrid_clock.cc:648] HybridClock initialized: now 1790577633698776 us; error 32 us; skew 500 ppm
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:33.701319 14721 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:40:33.702186 14721 webserver.cc:460] Webserver started at http://127.8.151.195:36347/ using document root <none> and password file <none>
I20260928 06:40:33.702700 14721 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:33.702772 14721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:33.702971 14721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:33.705034 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "b33fda8351b8498e9ccee787588673b6"
format_stamp: "Formatted at 2026-09-28 06:40:33 on dist-test-slave-gtr8"
server_key: "16b5dfd4d1eb0503405152f77669035f"
server_key_iv: "f3ecab5c6625d3da0f05ac6500711a15"
server_key_version: "encryptionkey@0"
I20260928 06:40:33.705546 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "b33fda8351b8498e9ccee787588673b6"
format_stamp: "Formatted at 2026-09-28 06:40:33 on dist-test-slave-gtr8"
server_key: "16b5dfd4d1eb0503405152f77669035f"
server_key_iv: "f3ecab5c6625d3da0f05ac6500711a15"
server_key_version: "encryptionkey@0"
I20260928 06:40:33.709151 14721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:33.711453 14736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:33.712620 14721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:40:33.712723 14721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "b33fda8351b8498e9ccee787588673b6"
format_stamp: "Formatted at 2026-09-28 06:40:33 on dist-test-slave-gtr8"
server_key: "16b5dfd4d1eb0503405152f77669035f"
server_key_iv: "f3ecab5c6625d3da0f05ac6500711a15"
server_key_version: "encryptionkey@0"
I20260928 06:40:33.712824 14721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:33.741636 14721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:33.744586 14721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:33.744773 14721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:33.745370 14721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:33.746220 14721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:33.746268 14721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:33.746320 14721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:33.746351 14721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:33.754529 14721 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:43437
I20260928 06:40:33.754558 14849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:43437 every 8 connection(s)
I20260928 06:40:33.755429 14721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:33.755625 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 14721
I20260928 06:40:33.755740 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:33.756063 8799 external_mini_cluster.cc:1428] Setting key 3c9ff5fefbc12f296a7b78dd5c432975
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:33.768328 14430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:33.757256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:56829 (local address 127.8.151.254:40263)
0928 06:40:33.757418 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:33.757423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:33.758287 (+ 864us) server_negotiation.cc:407] Connection header received
0928 06:40:33.759640 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:33.759642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:33.759687 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:33.759749 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:33.760811 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.761353 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.762487 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.762703 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.762788 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:33.765135 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:33.765138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:33.765140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:33.765163 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:33.766359 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.766839 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.766842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.766843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.766918 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.767278 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.767281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.767282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.767460 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:33.767546 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:33.768038 (+ 492us) server_negotiation.cc:299] Negotiation successful
0928 06:40:33.768153 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 06:40:33.769285 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:33.757707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:40263 (local address 127.8.151.195:56829)
0928 06:40:33.758143 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:33.758192 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:33.759472 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:33.759904 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:33.759910 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:33.760237 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:33.760691 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:33.760703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.761464 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.761467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:33.762367 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:33.762376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.762837 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.762841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:33.762996 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:33.763515 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:33.763532 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:33.764981 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:33.766471 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:33.766477 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:33.766491 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:33.766742 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:33.766995 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:33.766998 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:33.767000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:33.767127 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:33.767558 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:33.767565 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:33.767808 (+ 243us) client_negotiation.cc:769] Sending connection context
0928 06:40:33.767960 (+ 152us) client_negotiation.cc:240] Negotiation successful
0928 06:40:33.768270 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":98,"threads_started":1}
I20260928 06:40:33.770329 14850 heartbeater.cc:347] Connected to a master server at 127.8.151.254:40263
I20260928 06:40:33.770535 14850 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:33.770982 14850 heartbeater.cc:511] Master 127.8.151.254:40263 requested a full tablet report, sending...
I20260928 06:40:33.771914 14391 ts_manager.cc:194] Registered new tserver with Master: b33fda8351b8498e9ccee787588673b6 (127.8.151.195:43437)
I20260928 06:40:33.772455 14391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:56829
I20260928 06:40:33.781415 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/test-admin.keytab.
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:33.795993 8799 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260928 06:40:33.798661 14857 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:33.797350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35636 (local address 127.8.151.193:33071)
0928 06:40:33.797621 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:33.797635 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:33.797658 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:40:33.797707 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:33.797714 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:33.797900 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:33.798013 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:33.798501 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35636: BlockingRecv error: recv got EOF from 127.0.0.1:35636 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":87,"threads_started":1}
W20260928 06:40:33.798836 8799 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5cc)
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:33.799849 8799 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:33.800737 8799 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:40:33.809975 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:33.801597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.8.151.193:33071)
0928 06:40:33.801750 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:33.801755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:33.801819 (+ 64us) server_negotiation.cc:407] Connection header received
0928 06:40:33.801936 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:33.801939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:33.801982 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:33.802047 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:33.802598 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.803435 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.804229 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.804599 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.804692 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:33.806789 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:33.806798 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:33.806812 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:33.806836 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:33.808248 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.808656 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.808663 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.808665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.808752 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.809041 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.809044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.809046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.809289 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:33.809476 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:33.809681 (+ 205us) server_negotiation.cc:299] Negotiation successful
0928 06:40:33.809838 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260928 06:40:33.813501 14857 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:33.812581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.8.151.193:33071)
0928 06:40:33.812697 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:33.812702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:33.812782 (+ 80us) server_negotiation.cc:407] Connection header received
0928 06:40:33.812899 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:33.812902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:33.812941 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:33.812998 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:33.813405 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35662: BlockingRecv error: recv got EOF from 127.0.0.1:35662 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260928 06:40:33.813606 8799 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5cc' of type FILE
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:33.814524 8799 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:33.815373 8799 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:40:33.823383 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:33.816184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35670 (local address 127.8.151.193:33071)
0928 06:40:33.816295 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:33.816299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:33.816409 (+ 110us) server_negotiation.cc:407] Connection header received
0928 06:40:33.816520 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:33.816524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:33.816561 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:33.816621 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:33.817090 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.817657 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.818439 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.818663 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.818747 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:33.820767 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:33.820771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:33.820774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:33.820796 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:33.821934 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.822290 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.822293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.822295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.822391 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.822622 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.822625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.822627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.822780 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:33.822889 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:33.823104 (+ 215us) server_negotiation.cc:299] Negotiation successful
0928 06:40:33.823236 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260928 06:40:33.825793 14857 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:33.824883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35676 (local address 127.8.151.193:33071)
0928 06:40:33.824997 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:33.825000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:33.825051 (+ 51us) server_negotiation.cc:407] Connection header received
0928 06:40:33.825144 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:33.825146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:33.825178 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:33.825235 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:33.825719 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35676: BlockingRecv error: recv got EOF from 127.0.0.1:35676 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260928 06:40:33.825889 8799 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790577605970741-8799-0/krb5kdc/krb5cc' of type FILE
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:33.826756 8799 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:33.827610 8799 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:40:33.834774 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:33.828295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.8.151.193:33071)
0928 06:40:33.828408 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:33.828412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:33.828495 (+ 83us) server_negotiation.cc:407] Connection header received
0928 06:40:33.828587 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:33.828589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:33.828620 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:33.828669 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:33.829044 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.829515 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.830188 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:33.830379 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:33.830441 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:33.832312 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:33.832315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:33.832316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:33.832336 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:33.833410 (+ 1074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.833748 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.833750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.833751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.833812 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:33.834007 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:33.834009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:33.834010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:33.834146 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:33.834243 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:33.834522 (+ 279us) server_negotiation.cc:299] Negotiation successful
0928 06:40:33.834622 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 06:40:33.835693 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 14453
I20260928 06:40:33.840466 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 14587
I20260928 06:40:33.844671 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 14721
I20260928 06:40:33.848875 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 14359
2026-09-28T06:40:33Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2182 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14870](info): set up 2 sockets
Sep 28 06:40:33 dist-test-slave-gtr8 krb5kdc[14870](info): commencing operation
krb5kdc: starting...
W20260928 06:40:34.795773 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:34 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.254.keytab.
Entry for principal oryx/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.254.keytab.
I20260928 06:40:34.910467 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:33147
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.254.keytab
--principal=oryx/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:33147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:35.014504 14883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:35.014807 14883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:35.014855 14883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:35.018838 14883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:35.018893 14883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:35.018910 14883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:35.018939 14883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:35.018967 14883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:35.024124 14883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:33147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.254.keytab
--principal=oryx/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:33147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:35.025110 14883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:35.026098 14883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:35.034159 14891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:35.034175 14888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:35.034269 14883 server_base.cc:1034] running on GCE node
W20260928 06:40:35.034212 14889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:35.034917 14883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:35.035949 14883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:35.037109 14883 hybrid_clock.cc:648] HybridClock initialized: now 1790577635037082 us; error 36 us; skew 500 ppm
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:35.039680 14883 init.cc:377] Logged in from keytab as oryx/127.8.151.254@KRBTEST.COM (short username oryx)
I20260928 06:40:35.040562 14883 webserver.cc:460] Webserver started at http://127.8.151.254:41001/ using document root <none> and password file <none>
I20260928 06:40:35.041056 14883 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:35.041113 14883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:35.041308 14883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:35.043244 14883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "af8188b892fd4053a8fc54d272088786"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "232d79bbd098ee30ee6a6f2722494b50"
server_key_iv: "0b1de22d35c4d567351c3b63e06ee4bb"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.043757 14883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "af8188b892fd4053a8fc54d272088786"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "232d79bbd098ee30ee6a6f2722494b50"
server_key_iv: "0b1de22d35c4d567351c3b63e06ee4bb"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.047104 14883 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:35.049425 14898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.050614 14883 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:35.050733 14883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "af8188b892fd4053a8fc54d272088786"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "232d79bbd098ee30ee6a6f2722494b50"
server_key_iv: "0b1de22d35c4d567351c3b63e06ee4bb"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.050832 14883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:35.061908 14883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:35.064749 14883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:35.064939 14883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:35.071586 14883 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:33147
I20260928 06:40:35.071852 14950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:33147 every 8 connection(s)
I20260928 06:40:35.072499 14883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:35.075179 14951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:35.075515 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 14883
I20260928 06:40:35.075613 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:35.075866 8799 external_mini_cluster.cc:1428] Setting key 09075391fab2c41ac440450d0863617a
I20260928 06:40:35.081707 14951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786: Bootstrap starting.
I20260928 06:40:35.084306 14951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786: Neither blocks nor log segments found. Creating new log.
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:35.085023 14951 log.cc:824] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:35.086896 14951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786: No bootstrap required, opened a new log
I20260928 06:40:35.090809 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:35.077112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45730 (local address 127.8.151.254:33147)
0928 06:40:35.077495 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:35.077510 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:35.077545 (+ 35us) server_negotiation.cc:407] Connection header received
0928 06:40:35.078199 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:35.078216 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:35.078578 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:35.078881 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:35.079816 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.080835 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.081931 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.082405 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.082518 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:35.085136 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:35.085143 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:35.085160 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:35.085192 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:35.088145 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:35.088615 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:35.088619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:35.088624 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:35.088727 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:35.088977 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:35.088979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:35.088980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:35.089344 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:35.089486 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:35.089753 (+ 267us) server_negotiation.cc:299] Negotiation successful
0928 06:40:35.090014 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":101,"threads_started":1}
I20260928 06:40:35.090690 14951 raft_consensus.cc:348] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } }
I20260928 06:40:35.090921 14951 raft_consensus.cc:374] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:35.090973 14951 raft_consensus.cc:729] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af8188b892fd4053a8fc54d272088786, State: Initialized, Role: FOLLOWER
I20260928 06:40:35.091401 14951 consensus_queue.cc:260] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } }
I20260928 06:40:35.091539 14951 raft_consensus.cc:388] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:35.091621 14951 raft_consensus.cc:482] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:35.091729 14951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:35.092597 14951 raft_consensus.cc:504] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } }
I20260928 06:40:35.092917 14951 leader_election.cc:302] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af8188b892fd4053a8fc54d272088786; no voters:
I20260928 06:40:35.093201 14951 leader_election.cc:288] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:35.093297 14956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:35.093508 14956 raft_consensus.cc:686] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 1 LEADER]: Becoming Leader. State: Replica: af8188b892fd4053a8fc54d272088786, State: Running, Role: LEADER
I20260928 06:40:35.093784 14956 consensus_queue.cc:237] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } }
I20260928 06:40:35.094274 14951 sys_catalog.cc:564] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:35.095233 14958 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af8188b892fd4053a8fc54d272088786. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } }
I20260928 06:40:35.095353 14958 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:35.095674 14957 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } }
I20260928 06:40:35.095793 14957 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:35.095728 14965 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:35.098825 14965 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:35.104168 14965 catalog_manager.cc:1347] Generated new cluster ID: 9239a86ffe28446c88565c487ee3f0f2
I20260928 06:40:35.104233 14965 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:35.116578 14965 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:35.117470 14965 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:35.124115 14965 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786: Generated new TSK 0
I20260928 06:40:35.124922 14965 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab.
Entry for principal oryx/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab.
I20260928 06:40:35.158631 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:33147
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab
--principal=oryx/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:35.264916 14977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:35.265174 14977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:35.265234 14977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:35.269163 14977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:35.269213 14977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:35.269289 14977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:35.274343 14977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab
--principal=oryx/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:33147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:35.275357 14977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:35.276310 14977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:35.285655 14982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:35.285692 14983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:35.285748 14985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:35.285806 14977 server_base.cc:1034] running on GCE node
I20260928 06:40:35.286327 14977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:35.286944 14977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:35.288093 14977 hybrid_clock.cc:648] HybridClock initialized: now 1790577635288070 us; error 32 us; skew 500 ppm
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:35.290706 14977 init.cc:377] Logged in from keytab as oryx/127.8.151.193@KRBTEST.COM (short username oryx)
I20260928 06:40:35.291566 14977 webserver.cc:460] Webserver started at http://127.8.151.193:32847/ using document root <none> and password file <none>
I20260928 06:40:35.292071 14977 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:35.292128 14977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:35.292315 14977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:35.294242 14977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "c9ec552b32f84d568fa60fea398a060c"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "7c10de6842fa47e54e1a57a0ec041b08"
server_key_iv: "58cf85879f76ec9eef082583f07bcfeb"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.294783 14977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "c9ec552b32f84d568fa60fea398a060c"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "7c10de6842fa47e54e1a57a0ec041b08"
server_key_iv: "58cf85879f76ec9eef082583f07bcfeb"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.298382 14977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260928 06:40:35.300666 14992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.301821 14977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:35.301913 14977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "c9ec552b32f84d568fa60fea398a060c"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "7c10de6842fa47e54e1a57a0ec041b08"
server_key_iv: "58cf85879f76ec9eef082583f07bcfeb"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.302007 14977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:35.321971 14977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:35.325018 14977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:35.325183 14977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:35.325732 14977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:35.326515 14977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:35.326558 14977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.326601 14977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:35.326628 14977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.334887 14977 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:35507
I20260928 06:40:35.334918 15105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:35507 every 8 connection(s)
I20260928 06:40:35.335760 14977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:35.344642 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 14977
I20260928 06:40:35.344754 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:35.345010 8799 external_mini_cluster.cc:1428] Setting key 563af44268d06dcf64307d8ac62e3122
I20260928 06:40:35.348493 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:35.337425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:44125 (local address 127.8.151.254:33147)
0928 06:40:35.337562 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:35.337567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:35.338578 (+ 1011us) server_negotiation.cc:407] Connection header received
0928 06:40:35.339466 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:35.339469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:35.339512 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:35.339585 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:35.340668 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.341180 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.342370 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.342602 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.342678 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:35.344989 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:35.344992 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:35.344994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:35.345018 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:35.346562 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:35.347022 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:35.347025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:35.347027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:35.347105 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:35.347442 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:35.347445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:35.347446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:35.347621 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:35.347701 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:35.348208 (+ 507us) server_negotiation.cc:299] Negotiation successful
0928 06:40:35.348327 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 06:40:35.349298 15108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:35.337802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.193:44125)
0928 06:40:35.338443 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:35.338488 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:35.339282 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:35.339736 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:35.339742 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:35.340069 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:35.340562 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:35.340574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.341322 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.341324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:35.342253 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:35.342263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.342752 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.342756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:35.342904 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:35.343433 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:35.343448 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:35.344815 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:35.346667 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:35.346673 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:35.346688 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:35.346932 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:35.347209 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:35.347212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:35.347214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:35.347352 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:35.347711 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:35.347718 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:35.347961 (+ 243us) client_negotiation.cc:769] Sending connection context
0928 06:40:35.348165 (+ 204us) client_negotiation.cc:240] Negotiation successful
0928 06:40:35.348499 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":113,"threads_started":1}
I20260928 06:40:35.350361 15106 heartbeater.cc:347] Connected to a master server at 127.8.151.254:33147
I20260928 06:40:35.350605 15106 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:35.351192 15106 heartbeater.cc:511] Master 127.8.151.254:33147 requested a full tablet report, sending...
I20260928 06:40:35.352726 14915 ts_manager.cc:194] Registered new tserver with Master: c9ec552b32f84d568fa60fea398a060c (127.8.151.193:35507)
WARNING: no policy specified for oryx/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:40:35.353957 14915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:44125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab.
Entry for principal oryx/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab.
I20260928 06:40:35.372110 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:33147
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab
--principal=oryx/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:35.476608 15111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:35.476853 15111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:35.476910 15111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:35.481014 15111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:35.481084 15111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:35.481166 15111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:35.486373 15111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab
--principal=oryx/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:33147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:35.487388 15111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:35.488360 15111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:35.498257 15119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:35.498253 15116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:35.498545 15117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:35.498816 15111 server_base.cc:1034] running on GCE node
I20260928 06:40:35.499220 15111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:35.499935 15111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:35.501089 15111 hybrid_clock.cc:648] HybridClock initialized: now 1790577635501057 us; error 44 us; skew 500 ppm
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:35.503795 15111 init.cc:377] Logged in from keytab as oryx/127.8.151.194@KRBTEST.COM (short username oryx)
I20260928 06:40:35.504657 15111 webserver.cc:460] Webserver started at http://127.8.151.194:41337/ using document root <none> and password file <none>
I20260928 06:40:35.505203 15111 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:35.505267 15111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:35.505442 15111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:35.507498 15111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "2810e72ad0674063943bde466e9dc35f"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "d18d71d07a2e4b305afbb446d3d53ee2"
server_key_iv: "b3af687c0bdab90eb50407e104fafb05"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.508000 15111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "2810e72ad0674063943bde466e9dc35f"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "d18d71d07a2e4b305afbb446d3d53ee2"
server_key_iv: "b3af687c0bdab90eb50407e104fafb05"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.511530 15111 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:35.513821 15126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.514973 15111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:35.515071 15111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "2810e72ad0674063943bde466e9dc35f"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "d18d71d07a2e4b305afbb446d3d53ee2"
server_key_iv: "b3af687c0bdab90eb50407e104fafb05"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.515194 15111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:35.533710 15111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:35.536670 15111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:35.536839 15111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:35.537424 15111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:35.538199 15111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:35.538244 15111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.538282 15111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:35.538311 15111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.546525 15111 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:39851
I20260928 06:40:35.546554 15239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:39851 every 8 connection(s)
I20260928 06:40:35.547392 15111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:35.557683 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 15111
I20260928 06:40:35.557822 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:35.558138 8799 external_mini_cluster.cc:1428] Setting key fba75bfa5004611a70d19e6cf9ff14c8
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:35.562536 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:35.549108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:55661 (local address 127.8.151.254:33147)
0928 06:40:35.549239 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:35.549244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:35.550296 (+ 1052us) server_negotiation.cc:407] Connection header received
0928 06:40:35.551082 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:35.551084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:35.551128 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:35.551237 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:35.552316 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.552830 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.553894 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.554142 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.554213 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:35.559236 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:35.559239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:35.559241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:35.559266 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:35.560542 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:35.561127 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:35.561130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:35.561132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:35.561225 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:35.561523 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:35.561525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:35.561527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:35.561736 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:35.561838 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:35.562275 (+ 437us) server_negotiation.cc:299] Negotiation successful
0928 06:40:35.562383 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 06:40:35.563318 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:35.549534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.194:55661)
0928 06:40:35.550149 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:35.550189 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:35.550924 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:35.551418 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:35.551425 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:35.551757 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:35.552206 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:35.552219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.552921 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.552923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:35.553792 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:35.553800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.554273 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.554276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:35.554438 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:35.557474 (+ 3036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:35.557494 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:35.559069 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:35.560651 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:35.560657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:35.560671 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:35.561020 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:35.561310 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:35.561312 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:35.561313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:35.561445 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:35.561822 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:35.561827 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:35.562060 (+ 233us) client_negotiation.cc:769] Sending connection context
0928 06:40:35.562208 (+ 148us) client_negotiation.cc:240] Negotiation successful
0928 06:40:35.562448 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":109,"threads_started":1}
I20260928 06:40:35.564361 15240 heartbeater.cc:347] Connected to a master server at 127.8.151.254:33147
I20260928 06:40:35.564585 15240 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:35.565032 15240 heartbeater.cc:511] Master 127.8.151.254:33147 requested a full tablet report, sending...
I20260928 06:40:35.566017 14915 ts_manager.cc:194] Registered new tserver with Master: 2810e72ad0674063943bde466e9dc35f (127.8.151.194:39851)
I20260928 06:40:35.566573 14915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:55661
WARNING: no policy specified for oryx/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab.
Entry for principal oryx/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab.
I20260928 06:40:35.581975 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:33147
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab
--principal=oryx/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:35.687512 15245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:35.687758 15245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:35.687813 15245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:35.691700 15245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:35.691751 15245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:35.691823 15245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:35.696995 15245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab
--principal=oryx/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:33147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:35.697933 15245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:35.698866 15245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:35.708124 15250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:35.708137 15251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:35.708351 15245 server_base.cc:1034] running on GCE node
W20260928 06:40:35.708168 15253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:35.708779 15245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:35.709368 15245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:35.710515 15245 hybrid_clock.cc:648] HybridClock initialized: now 1790577635710493 us; error 31 us; skew 500 ppm
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:35.713027 15245 init.cc:377] Logged in from keytab as oryx/127.8.151.195@KRBTEST.COM (short username oryx)
I20260928 06:40:35.713796 15245 webserver.cc:460] Webserver started at http://127.8.151.195:41049/ using document root <none> and password file <none>
I20260928 06:40:35.714318 15245 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:35.714375 15245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:35.714561 15245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:35.716557 15245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "b4c77749c17942f3bece23898e90320c"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "2ea32a0289152db1da98ff366c739b76"
server_key_iv: "a9dfa29e1c1e1b12de7f440e187b2b31"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.717073 15245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "b4c77749c17942f3bece23898e90320c"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "2ea32a0289152db1da98ff366c739b76"
server_key_iv: "a9dfa29e1c1e1b12de7f440e187b2b31"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.720749 15245 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:35.723199 15260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.724468 15245 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:35.724565 15245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "b4c77749c17942f3bece23898e90320c"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "2ea32a0289152db1da98ff366c739b76"
server_key_iv: "a9dfa29e1c1e1b12de7f440e187b2b31"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.724686 15245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:35.740671 15245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:35.743893 15245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:35.744072 15245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:35.744680 15245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:35.745527 15245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:35.745572 15245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.745612 15245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:35.745638 15245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.754664 15245 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:46477
I20260928 06:40:35.754717 15373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:46477 every 8 connection(s)
I20260928 06:40:35.755631 15245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:35.757513 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 15245
I20260928 06:40:35.757614 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:35.757879 8799 external_mini_cluster.cc:1428] Setting key 04890028a33f079bf0b2d51c4659b15c
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:35.769286 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:35.757462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:57351 (local address 127.8.151.254:33147)
0928 06:40:35.757620 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:35.757625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:35.759053 (+ 1428us) server_negotiation.cc:407] Connection header received
0928 06:40:35.760159 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:35.760161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:35.760207 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:35.760265 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:35.761640 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.762178 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.763324 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.763552 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.763632 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:35.765937 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:35.765940 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:35.765942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:35.765967 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:35.767323 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:35.767821 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:35.767825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:35.767826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:35.767908 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:35.768222 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:35.768225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:35.768227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:35.768414 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:35.768498 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:35.768997 (+ 499us) server_negotiation.cc:299] Negotiation successful
0928 06:40:35.769122 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 06:40:35.770115 15376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:35.757937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.195:57351)
0928 06:40:35.758919 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:35.758963 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:35.759978 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:35.760427 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:35.760434 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:35.760900 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:35.761517 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:35.761533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.762266 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.762269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:35.763222 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:35.763233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.763671 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:35.763674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:35.763815 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:35.764313 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:35.764330 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:35.765770 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:35.767434 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:35.767441 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:35.767458 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:35.767718 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:35.767990 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:35.767993 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:35.767995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:35.768134 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:35.768511 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:35.768518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:35.768765 (+ 247us) client_negotiation.cc:769] Sending connection context
0928 06:40:35.768937 (+ 172us) client_negotiation.cc:240] Negotiation successful
0928 06:40:35.769271 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":600,"threads_started":1}
I20260928 06:40:35.771467 15374 heartbeater.cc:347] Connected to a master server at 127.8.151.254:33147
I20260928 06:40:35.771730 15374 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:35.772253 15374 heartbeater.cc:511] Master 127.8.151.254:33147 requested a full tablet report, sending...
I20260928 06:40:35.773341 14915 ts_manager.cc:194] Registered new tserver with Master: b4c77749c17942f3bece23898e90320c (127.8.151.195:46477)
I20260928 06:40:35.773964 14915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:57351
I20260928 06:40:35.784086 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.151.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.253.keytab.
Entry for principal oryx/127.8.151.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.253.keytab.
I20260928 06:40:35.808419 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.253:44903
--webserver_interface=127.8.151.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.253.keytab
--principal=oryx/127.8.151.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:33147,127.8.151.253:44903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:35.918802 15379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:35.919039 15379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:35.919091 15379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:35.922988 15379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:35.923043 15379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:35.923060 15379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:35.923074 15379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:35.923089 15379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:35.928222 15379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:33147,127.8.151.253:44903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.253.keytab
--principal=oryx/127.8.151.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.253:44903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.151.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
W20260928 06:40:35.928404 15379 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.151.254:33147,127.8.151.253:44903'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260928 06:40:35.929246 15379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:35.930186 15379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:35.938349 15384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:35.938421 15379 server_base.cc:1034] running on GCE node
W20260928 06:40:35.938354 15387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:35.938575 15385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:35.939079 15379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:35.940138 15379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:35.941294 15379 hybrid_clock.cc:648] HybridClock initialized: now 1790577635941276 us; error 28 us; skew 500 ppm
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:35.943792 15379 init.cc:377] Logged in from keytab as oryx/127.8.151.253@KRBTEST.COM (short username oryx)
I20260928 06:40:35.944612 15379 webserver.cc:460] Webserver started at http://127.8.151.253:37977/ using document root <none> and password file <none>
I20260928 06:40:35.945119 15379 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:35.945181 15379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:35.945376 15379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:35.947329 15379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data/instance:
uuid: "f177e7daf9234317ba18e3de53eef1b7"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "08363acf1eb98d447835143037e7e9d7"
server_key_iv: "29f728e028eed3496fbe29307fa15996"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.947889 15379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal/instance:
uuid: "f177e7daf9234317ba18e3de53eef1b7"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "08363acf1eb98d447835143037e7e9d7"
server_key_iv: "29f728e028eed3496fbe29307fa15996"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.951460 15379 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:40:35.953684 15394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:35.955055 15379 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:35.955190 15379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal
uuid: "f177e7daf9234317ba18e3de53eef1b7"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "08363acf1eb98d447835143037e7e9d7"
server_key_iv: "29f728e028eed3496fbe29307fa15996"
server_key_version: "encryptionkey@0"
I20260928 06:40:35.955282 15379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:35.979818 15379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:35.982842 15379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:35.982998 15379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:35.989356 15379 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.253:44903
I20260928 06:40:35.989600 15446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.253:44903 every 8 connection(s)
I20260928 06:40:35.990280 15379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
I20260928 06:40:35.992913 15447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:35.994524 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 15379
I20260928 06:40:35.996886 15447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.151.253@KRBTEST.COM
Sep 28 06:40:35 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.253@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:36.007458 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:35.998656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45736 (local address 127.8.151.254:33147)
0928 06:40:35.998784 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:35.998788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:35.999076 (+ 288us) server_negotiation.cc:407] Connection header received
0928 06:40:35.999229 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:35.999232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:35.999280 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:35.999372 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:35.999968 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.000490 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.001433 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.001638 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.001719 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.004206 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.004209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.004211 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.004233 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.005662 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.006135 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.006138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.006140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.006216 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.006470 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.006473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.006475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.006732 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.006822 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.007240 (+ 418us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.007332 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:40:36.008001 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:35.998595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:45736)
0928 06:40:35.998957 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:35.999003 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:35.999112 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:35.999422 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:35.999426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:35.999724 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:35.999848 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:35.999855 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.000604 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.000607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.001315 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.001324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.001771 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.001775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.001875 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.002330 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.002344 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.004063 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.005765 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.005774 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.005780 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.006058 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.006300 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.006302 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.006303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.006379 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.006831 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.006838 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.006980 (+ 142us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.007594 (+ 614us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.007765 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":20,"spinlock_wait_cycles":2944,"thread_start_us":147,"threads_started":1}
I20260928 06:40:36.008006 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:35.995999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59968 (local address 127.8.151.253:44903)
0928 06:40:35.996361 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:35.996380 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:35.996421 (+ 41us) server_negotiation.cc:407] Connection header received
0928 06:40:35.997049 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:35.997065 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:35.997381 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:35.997707 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:35.998522 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:35.999737 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.000608 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.001145 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.001252 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.003697 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.003703 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.003716 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.003740 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.005331 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.005725 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.005732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.005738 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.005853 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.006130 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.006134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.006135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.006580 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.006782 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.006978 (+ 196us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.007238 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":230,"spinlock_wait_cycles":1280,"thread_start_us":90,"threads_started":1}
I20260928 06:40:36.009199 15447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.253@KRBTEST.COM for oryx/127.8.151.253@KRBTEST.COM
I20260928 06:40:36.016479 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.009614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:44903 (local address 127.0.0.1:59980)
0928 06:40:36.009724 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.009738 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.009807 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.009993 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.009995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.010125 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.010201 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.010206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.011189 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.011192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.011668 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.011675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.012052 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.012055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.012175 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.012568 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.012578 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.013719 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.015346 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.015348 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.015350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.015508 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.015701 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.015703 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.015704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.015761 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.016132 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.016134 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.016200 (+ 66us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.016277 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.016361 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":30}
I20260928 06:40:36.016621 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.009716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59980 (local address 127.8.151.253:44903)
0928 06:40:36.009788 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.009792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.009803 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:40:36.009878 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.009881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.009919 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.009974 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.010293 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.011070 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.011760 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.011968 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.012046 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.013863 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.013866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.013868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.013887 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.015262 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.015575 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.015577 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.015578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.015645 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.015865 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.015868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.015869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.016048 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.016140 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.016390 (+ 250us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.016486 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":25}
I20260928 06:40:36.019126 15447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Bootstrap starting.
I20260928 06:40:36.021338 15447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:36.021972 15447 log.cc:824] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:36.023715 15447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: No bootstrap required, opened a new log
I20260928 06:40:36.026500 15447 raft_consensus.cc:348] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } }
I20260928 06:40:36.026684 15447 raft_consensus.cc:374] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:36.026721 15447 raft_consensus.cc:729] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f177e7daf9234317ba18e3de53eef1b7, State: Initialized, Role: FOLLOWER
I20260928 06:40:36.027089 15447 consensus_queue.cc:260] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } }
I20260928 06:40:36.027549 15453 sys_catalog.cc:455] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } } }
I20260928 06:40:36.027657 15453 sys_catalog.cc:458] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:40:36.028236 15447 sys_catalog.cc:564] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 06:40:36.032577 15464 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:40:36.032639 15464 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:40:36.034555 15379 master_runner.cc:428] Detected that this master f177e7daf9234317ba18e3de53eef1b7 is joining an existing cluster
I20260928 06:40:36.034600 15379 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.151.253:44903
I20260928 06:40:36.043848 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.036386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:44903 (local address 127.0.0.1:59992)
0928 06:40:36.036751 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.036764 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.036832 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.037044 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.037047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.037218 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.037313 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.037318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.038112 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.038116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.039068 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.039079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.039720 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.039722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.039803 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.040281 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.040295 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.040600 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.042317 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.042321 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.042323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.042553 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.042820 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.042823 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.042825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.042911 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.043305 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.043309 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.043395 (+ 86us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.043502 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.043635 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":24,"thread_start_us":90,"threads_started":1}
I20260928 06:40:36.043874 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.036711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59992 (local address 127.8.151.253:44903)
0928 06:40:36.036781 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.036786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.036819 (+ 33us) server_negotiation.cc:407] Connection header received
0928 06:40:36.036910 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.036913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.036949 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.037006 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.037405 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.037998 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.039231 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.039543 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.039627 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.040713 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.040716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.040719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.040740 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.042223 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.042647 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.042651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.042653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.042730 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.043003 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.043007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.043009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.043217 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.043306 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.043555 (+ 249us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.043709 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":26}
I20260928 06:40:36.044553 15470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.036360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:45738)
0928 06:40:36.036730 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.036848 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.036959 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.037300 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.037303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.037512 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.037628 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.037636 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.038459 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.038462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.039051 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.039059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.039598 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.039602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.039711 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.040777 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.040792 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.041220 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.042847 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.042850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.042852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.043041 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.043360 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.043362 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.043364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.043439 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.044138 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.044140 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.044205 (+ 65us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.044287 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.044397 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":64,"thread_start_us":90,"threads_started":1}
I20260928 06:40:36.044613 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.036441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45738 (local address 127.8.151.254:33147)
0928 06:40:36.036556 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.036561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.036898 (+ 337us) server_negotiation.cc:407] Connection header received
0928 06:40:36.037059 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.037062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.037109 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.037157 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.037747 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.038365 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.039168 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.039488 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.039590 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.041355 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.041359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.041361 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.041389 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.042749 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.043178 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.043181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.043182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.043261 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.043624 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.043627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.043628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.043826 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.044100 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.044358 (+ 258us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.044479 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.254@KRBTEST.COM for oryx/127.8.151.253@KRBTEST.COM
I20260928 06:40:36.056684 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.048357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60002 (local address 127.8.151.253:44903)
0928 06:40:36.048567 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.048571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.048780 (+ 209us) server_negotiation.cc:407] Connection header received
0928 06:40:36.048948 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.048951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.048990 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.049049 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.049786 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.050234 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.051451 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.051646 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.051719 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.053891 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.053894 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.053897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.053921 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.055159 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.055672 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.055675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.055676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.055741 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.055975 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.055977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.055978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.056114 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.056178 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.056505 (+ 327us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.056580 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260928 06:40:36.056825 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.048305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:44903 (local address 127.0.0.1:60002)
0928 06:40:36.048654 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.048708 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.048834 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.049092 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.049097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.049525 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.049665 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.049672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.050326 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.050330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.051333 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.051344 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.051737 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.051740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.051852 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.052277 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.052298 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.053777 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.055261 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.055271 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.055277 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.055588 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.055815 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.055818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.055820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.055907 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.056195 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.056201 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.056331 (+ 130us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.056475 (+ 144us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.056653 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":95,"threads_started":1}
I20260928 06:40:36.058143 14915 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.151.253:44903: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7"
member_type: NON_VOTER
last_known_addr {
host: "127.8.151.253"
port: 44903
}
attrs {
promote: true
}
}
dest_uuid: "af8188b892fd4053a8fc54d272088786"
cas_config_opid_index: -1
I20260928 06:40:36.058738 14915 consensus_queue.cc:237] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } }
I20260928 06:40:36.060357 15473 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f177e7daf9234317ba18e3de53eef1b7 (127.8.151.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } }
I20260928 06:40:36.061054 15421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:36.061793 15473 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.151.253:44903
I20260928 06:40:36.062176 15379 master.cc:537] Master@127.8.151.253:44903 shutting down...
I20260928 06:40:36.062225 15474 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } }
I20260928 06:40:36.062311 15474 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:36.062319 15421 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Refusing update from remote peer af8188b892fd4053a8fc54d272088786: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260928 06:40:36.062881 15474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 06:40:36.065407 14899 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260928 06:40:36.066280 14899 consensus_peers.cc:487] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 -> Peer f177e7daf9234317ba18e3de53eef1b7 (127.8.151.253:44903): Couldn't send request to peer f177e7daf9234317ba18e3de53eef1b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 06:40:36.067281 15379 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:36.067561 15379 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:36.067662 15379 tablet_replica.cc:330] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: stopping tablet replica
I20260928 06:40:36.071764 15379 master.cc:552] Master@127.8.151.253:44903 shutdown complete.
I20260928 06:40:36.071803 15379 master_runner.cc:305] Clearing existing system tablet
I20260928 06:40:36.072000 15379 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:36.072045 15379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:36.073024 15379 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 06:40:36.073716 15479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.073904 15379 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:36.074021 15379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:36.074059 15379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal
uuid: "f177e7daf9234317ba18e3de53eef1b7"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "08363acf1eb98d447835143037e7e9d7"
server_key_iv: "29f728e028eed3496fbe29307fa15996"
server_key_version: "encryptionkey@0"
I20260928 06:40:36.075673 15379 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:36.077471 15379 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 06:40:36.077548 15379 log.cc:1196] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260928 06:40:36.077744 15379 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Deleting consensus metadata
I20260928 06:40:36.077888 15379 master_runner.cc:315] Copying system tablet from 127.8.151.254:33147
I20260928 06:40:36.078855 15379 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: tablet copy: Beginning tablet copy session from remote peer at address 127.8.151.254:33147
I20260928 06:40:36.086037 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.079644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45744 (local address 127.8.151.254:33147)
0928 06:40:36.079750 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.079754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.079766 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:36.079800 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.079802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.079840 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.079901 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.080314 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.080829 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.081602 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.081823 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.081905 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.082873 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.082877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.082879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.082905 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.084249 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.084717 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.084721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.084723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.084807 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.085116 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.085120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.085121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.085334 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.085405 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.085772 (+ 367us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.085883 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 06:40:36.086709 15485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.079325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:45744)
0928 06:40:36.079564 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.079578 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.079678 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.079931 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.079933 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.080117 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.080208 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.080214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.080925 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.080928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.081497 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.081505 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.081914 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.081918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.082006 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.082441 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.082452 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.082748 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.084349 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.084353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.084355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.084599 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.084919 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.084922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.084924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.085011 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.085495 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.085498 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.085594 (+ 96us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.086231 (+ 637us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.086372 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":80,"threads_started":1}
I20260928 06:40:36.087339 14935 tablet_copy_service.cc:140] P af8188b892fd4053a8fc54d272088786: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f177e7daf9234317ba18e3de53eef1b7 ({username='oryx', principal='oryx/127.8.151.253@KRBTEST.COM'} at 127.0.0.1:45744)
I20260928 06:40:36.087510 14935 tablet_copy_service.cc:161] P af8188b892fd4053a8fc54d272088786: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f177e7daf9234317ba18e3de53eef1b7 at {username='oryx', principal='oryx/127.8.151.253@KRBTEST.COM'} at 127.0.0.1:45744: session id = f177e7daf9234317ba18e3de53eef1b7-00000000000000000000000000000000
WARNING: no policy specified for oryx/127.8.151.252@KRBTEST.COM; defaulting to no policy
I20260928 06:40:36.089174 14935 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 06:40:36.090767 15379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.252@KRBTEST.COM" created.
I20260928 06:40:36.092972 15379 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: tablet copy: Starting download of 0 data blocks...
I20260928 06:40:36.093137 15379 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: tablet copy: Starting download of 1 WAL segments...
I20260928 06:40:36.094602 15379 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 06:40:36.096302 14935 tablet_copy_service.cc:342] P af8188b892fd4053a8fc54d272088786: Request end of tablet copy session f177e7daf9234317ba18e3de53eef1b7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.151.253@KRBTEST.COM'} at 127.0.0.1:45744
I20260928 06:40:36.096400 14935 tablet_copy_service.cc:433] P af8188b892fd4053a8fc54d272088786: ending tablet copy session f177e7daf9234317ba18e3de53eef1b7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f177e7daf9234317ba18e3de53eef1b7
I20260928 06:40:36.103116 15379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:36.104962 15487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:36.105065 15379 server_base.cc:1034] running on GCE node
W20260928 06:40:36.105209 15490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:36.105381 15488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:36.105594 15379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:36.105906 15379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:36.107062 15379 hybrid_clock.cc:648] HybridClock initialized: now 1790577636107049 us; error 29 us; skew 500 ppm
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.252.keytab.
Entry for principal oryx/127.8.151.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.252.keytab.
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:36.108587 15379 init.cc:377] Logged in from keytab as oryx/127.8.151.253@KRBTEST.COM (short username oryx)
I20260928 06:40:36.108605 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.252:43149
--webserver_interface=127.8.151.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.252.keytab
--principal=oryx/127.8.151.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:33147,127.8.151.253:44903,127.8.151.252:43149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:40:36.109025 15379 webserver.cc:460] Webserver started at http://127.8.151.253:35985/ using document root <none> and password file <none>
I20260928 06:40:36.109200 15379 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:36.109241 15379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:36.110467 15379 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:36.111581 15498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.111816 15379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:36.111886 15379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal
uuid: "f177e7daf9234317ba18e3de53eef1b7"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "08363acf1eb98d447835143037e7e9d7"
server_key_iv: "29f728e028eed3496fbe29307fa15996"
server_key_version: "encryptionkey@0"
I20260928 06:40:36.111968 15379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:36.119678 15379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:36.121443 15379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:36.125789 15379 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.253:44903
I20260928 06:40:36.126022 15550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.253:44903 every 8 connection(s)
I20260928 06:40:36.126392 15379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
I20260928 06:40:36.128146 15551 sys_catalog.cc:263] Verifying existing consensus state
I20260928 06:40:36.129189 15551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Bootstrap starting.
I20260928 06:40:36.139132 15551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:40:36.139468 15551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Bootstrap complete.
I20260928 06:40:36.140029 15551 raft_consensus.cc:348] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } }
I20260928 06:40:36.140164 15551 raft_consensus.cc:729] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f177e7daf9234317ba18e3de53eef1b7, State: Initialized, Role: LEARNER
I20260928 06:40:36.140269 15551 consensus_queue.cc:260] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } }
I20260928 06:40:36.140617 15554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } }
I20260928 06:40:36.140743 15551 sys_catalog.cc:564] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:36.140785 15554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: This master's current role is: LEARNER
I20260928 06:40:36.143507 15565 catalog_manager.cc:1259] Loaded cluster ID: 9239a86ffe28446c88565c487ee3f0f2
I20260928 06:40:36.143618 15565 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: loading cluster ID for follower catalog manager: success
I20260928 06:40:36.145654 15565 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: acquiring CA information for follower catalog manager: success
I20260928 06:40:36.146950 15565 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260928 06:40:36.225600 15496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:36.225865 15496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:36.225911 15496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:36.229905 15496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:36.229960 15496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:36.229977 15496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:36.229995 15496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:36.230022 15496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:36.235459 15496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:33147,127.8.151.253:44903,127.8.151.252:43149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.252.keytab
--principal=oryx/127.8.151.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.252:43149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.151.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:36.236492 15496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:36.237501 15496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:36.245940 15571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:36.245965 15573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:36.245957 15570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:36.246263 15496 server_base.cc:1034] running on GCE node
I20260928 06:40:36.246793 15496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:36.247869 15496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:36.249030 15496 hybrid_clock.cc:648] HybridClock initialized: now 1790577636249012 us; error 31 us; skew 500 ppm
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:36.251622 15496 init.cc:377] Logged in from keytab as oryx/127.8.151.252@KRBTEST.COM (short username oryx)
I20260928 06:40:36.252449 15496 webserver.cc:460] Webserver started at http://127.8.151.252:34515/ using document root <none> and password file <none>
I20260928 06:40:36.252964 15496 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:36.253022 15496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:36.253201 15496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:36.255362 15496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data/instance:
uuid: "6a5be8b6352846d5a16b1c6a50b155d0"
format_stamp: "Formatted at 2026-09-28 06:40:36 on dist-test-slave-gtr8"
server_key: "ceeb62d85cbf627e3ee48183d65b8e9e"
server_key_iv: "807be7da8b68ad6b74f6dd8ec4093c8b"
server_key_version: "encryptionkey@0"
I20260928 06:40:36.255889 15496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal/instance:
uuid: "6a5be8b6352846d5a16b1c6a50b155d0"
format_stamp: "Formatted at 2026-09-28 06:40:36 on dist-test-slave-gtr8"
server_key: "ceeb62d85cbf627e3ee48183d65b8e9e"
server_key_iv: "807be7da8b68ad6b74f6dd8ec4093c8b"
server_key_version: "encryptionkey@0"
I20260928 06:40:36.259498 15496 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:36.261871 15580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.263047 15496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:36.263193 15496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal
uuid: "6a5be8b6352846d5a16b1c6a50b155d0"
format_stamp: "Formatted at 2026-09-28 06:40:36 on dist-test-slave-gtr8"
server_key: "ceeb62d85cbf627e3ee48183d65b8e9e"
server_key_iv: "807be7da8b68ad6b74f6dd8ec4093c8b"
server_key_version: "encryptionkey@0"
I20260928 06:40:36.263290 15496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:36.287004 15496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:36.289750 15496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:36.289911 15496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:36.296453 15496 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.252:43149
I20260928 06:40:36.296676 15632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.252:43149 every 8 connection(s)
I20260928 06:40:36.297346 15496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
I20260928 06:40:36.300091 15633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:36.303857 15633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:40:36.305545 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 15496
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.252@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.151.252@KRBTEST.COM
I20260928 06:40:36.317531 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.305696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45748 (local address 127.8.151.254:33147)
0928 06:40:36.305812 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.305816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.307006 (+ 1190us) server_negotiation.cc:407] Connection header received
0928 06:40:36.307642 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.307646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.307694 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.307790 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.308805 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.309645 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.311015 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.311369 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.311488 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.314075 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.314078 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.314080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.314105 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.315598 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.316106 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.316109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.316111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.316199 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.316484 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.316487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.316489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.316665 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.316747 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.317236 (+ 489us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.317355 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 06:40:36.318390 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.306163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:45748)
0928 06:40:36.306863 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.306910 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.307459 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.307876 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.307882 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.308334 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.308677 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.308693 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.309775 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.309778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.310913 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.310923 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.311522 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.311525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.311663 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.312371 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.312387 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.313931 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.315700 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.315707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.315715 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.316012 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.316266 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.316269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.316271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.316397 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.316759 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.316766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.317021 (+ 255us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.317179 (+ 158us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.317482 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"spinlock_wait_cycles":9600,"thread_start_us":159,"threads_started":1}
I20260928 06:40:36.318390 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.306364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.8.151.252:43149)
0928 06:40:36.306687 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.306701 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.306732 (+ 31us) server_negotiation.cc:407] Connection header received
0928 06:40:36.307452 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.307470 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.307680 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.307838 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.308679 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.309882 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.310989 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.311560 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.311660 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.314469 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.314476 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.314490 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.314512 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.316105 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.316546 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.316552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.316554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.316626 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.316873 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.316876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.316878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.317094 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.317272 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.317491 (+ 219us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.317880 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":214,"spinlock_wait_cycles":1792,"thread_start_us":128,"threads_started":1}
I20260928 06:40:36.319552 15633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.252@KRBTEST.COM for oryx/127.8.151.253@KRBTEST.COM
I20260928 06:40:36.326706 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.319971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:44903 (local address 127.0.0.1:60016)
0928 06:40:36.320102 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.320112 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.320167 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.320637 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.320639 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.320755 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.320856 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.320860 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.321699 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.321702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.322243 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.322251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.322654 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.322657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.322732 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.323125 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.323156 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.324240 (+ 1084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.325685 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.325687 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.325689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.325863 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.326054 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.326055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.326057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.326115 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.326376 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.326378 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.326444 (+ 66us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.326511 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.326598 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260928 06:40:36.326766 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.320088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.8.151.253:44903)
0928 06:40:36.320458 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.320462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.320473 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:40:36.320507 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.320509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.320546 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.320612 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.320939 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.321596 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.322330 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.322549 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.322615 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.324400 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.324403 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.324405 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.324429 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.325598 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.325930 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.325933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.325934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.325996 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.326171 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.326173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.326174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.326316 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.326381 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.326555 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.326644 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":115,"threads_started":1}
I20260928 06:40:36.327380 15633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.252@KRBTEST.COM for oryx/127.8.151.252@KRBTEST.COM
I20260928 06:40:36.334010 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.327796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:43149 (local address 127.0.0.1:48388)
0928 06:40:36.327884 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.327894 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.327946 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.328126 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.328128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.328235 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.328295 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.328299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.329025 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.329028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.329538 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.329546 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.329936 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.329939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.329993 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.330373 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.330383 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.331527 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.332766 (+ 1239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.332769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.332771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.332940 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.333154 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.333157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.333159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.333226 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.333530 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.333533 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.333609 (+ 76us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.333687 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.333808 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 06:40:36.334021 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.327889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.8.151.252:43149)
0928 06:40:36.327982 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.327986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.327998 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:36.328023 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.328024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.328054 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.328111 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.328390 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.328913 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.329635 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.329858 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.329932 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.331637 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.331640 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.331643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.331664 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.332677 (+ 1013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.333017 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.333019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.333021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.333083 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.333297 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.333299 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.333301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.333453 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.333530 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.333736 (+ 206us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.333857 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":34}
I20260928 06:40:36.337075 15633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: Bootstrap starting.
I20260928 06:40:36.339231 15633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:36.339933 15633 log.cc:824] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:36.341765 15633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: No bootstrap required, opened a new log
I20260928 06:40:36.344642 15633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } }
I20260928 06:40:36.344823 15633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:36.344856 15633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5be8b6352846d5a16b1c6a50b155d0, State: Initialized, Role: FOLLOWER
I20260928 06:40:36.345245 15633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } }
I20260928 06:40:36.345641 15639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } } }
I20260928 06:40:36.345754 15639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:40:36.346305 15633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 06:40:36.350625 15650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:40:36.350708 15650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:40:36.353747 15496 master_runner.cc:428] Detected that this master 6a5be8b6352846d5a16b1c6a50b155d0 is joining an existing cluster
I20260928 06:40:36.353794 15496 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.151.252:43149
I20260928 06:40:36.356383 15106 heartbeater.cc:503] Master 127.8.151.254:33147 was elected leader, sending a full tablet report...
I20260928 06:40:36.371471 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 14977
I20260928 06:40:36.377832 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:35507
--local_ip_for_outbound_sockets=127.8.151.193
--tserver_master_addrs=127.8.151.254:33147
--webserver_port=32847
--webserver_interface=127.8.151.193
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab
--principal=oryx/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.151.254:33147,127.8.151.253:44903
--tserver_master_addrs=127.8.151.254:33147,127.8.151.253:44903,127.8.151.252:43149 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:40:36.396121 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.356155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60030 (local address 127.8.151.253:44903)
0928 06:40:36.357011 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.357015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.357026 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:40:36.357059 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.357062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.357104 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.357170 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.358094 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.358873 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.361030 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.361248 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.361303 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.392065 (+ 30762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.392070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.392073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.392104 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.394135 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.394657 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.394660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.394662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.394745 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.395059 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.395063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.395065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.395327 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.395393 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.395817 (+ 424us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.395950 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777}
I20260928 06:40:36.396636 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.355636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:45760)
0928 06:40:36.355922 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.355948 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.356036 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.357283 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.357286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.357485 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.357593 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.357599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.358635 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.358639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.359999 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.360010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.360559 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.360563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.360656 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.361043 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.361057 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.392095 (+ 31038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.394973 (+ 2878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.394988 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.394990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.395253 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.395575 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.395578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.395580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.395676 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.396111 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.396114 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.396208 (+ 94us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.396314 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.396461 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260928 06:40:36.397058 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.355665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:44903 (local address 127.0.0.1:60030)
0928 06:40:36.356279 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.356291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.356367 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.357711 (+ 1344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.357714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.357886 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.357979 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.357985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.358997 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.359000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.359861 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.359869 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.361427 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.361429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.361493 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.361883 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.361893 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.391823 (+ 29930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.394258 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.394261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.394264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.394542 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.394853 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.394856 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.394858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.394942 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.395471 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.395473 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.395631 (+ 158us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.396574 (+ 943us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.396902 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":85,"threads_started":1}
I20260928 06:40:36.397233 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.356038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.8.151.254:33147)
0928 06:40:36.356682 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.356686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.356699 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:36.356738 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.356740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.356785 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.356856 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.357708 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.358507 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.360128 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.360443 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.360544 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.392492 (+ 31948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.392498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.392501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.392537 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.394858 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.395390 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.395393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.395395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.395480 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.395768 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.395771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.395772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.395986 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.396070 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.396976 (+ 906us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.397090 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560}
I20260928 06:40:36.400038 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.356295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48392 (local address 127.8.151.252:43149)
0928 06:40:36.356383 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.356388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.356494 (+ 106us) server_negotiation.cc:407] Connection header received
0928 06:40:36.356759 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.356761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.356796 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.356864 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.357443 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.358138 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.359170 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.359485 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.359553 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.360809 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.360814 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.360816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.360848 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.394140 (+ 33292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.398900 (+ 4760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.398903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.398905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.398974 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.399239 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.399242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.399244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.399425 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.399506 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.399806 (+ 300us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.399913 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":27}
I20260928 06:40:36.400653 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.356042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:43149 (local address 127.0.0.1:48392)
0928 06:40:36.356422 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.356434 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.356543 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.356968 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.356971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.357215 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.357337 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.357345 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.358235 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.358237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.359051 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.359058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.359684 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.359687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.359765 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.360242 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.360255 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.360706 (+ 451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.398580 (+ 37874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.398583 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.398585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.398804 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.399061 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.399064 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.399065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.399159 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.399568 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.399570 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.399646 (+ 76us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.400363 (+ 717us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.400505 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":89,"threads_started":1}
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577635, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.254@KRBTEST.COM for oryx/127.8.151.252@KRBTEST.COM
I20260928 06:40:36.407904 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.401378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48402 (local address 127.8.151.252:43149)
0928 06:40:36.401475 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.401478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.401490 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:36.401544 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.401547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.401593 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.401654 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.402141 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.402607 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.403427 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.403632 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.403715 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.405374 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.405377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.405379 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.405398 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.406530 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.406890 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.406893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.406895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.406968 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.407214 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.407217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.407219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.407361 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.407442 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.407683 (+ 241us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.407769 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:40:36.407910 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.400936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:43149 (local address 127.0.0.1:48402)
0928 06:40:36.401176 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.401190 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.401261 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.401765 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.401768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.401927 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.402018 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.402024 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.402705 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.402707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.403327 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.403337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.403724 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.403728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.403813 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.404164 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.404175 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.405257 (+ 1082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.406629 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.406632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.406635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.406806 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.407047 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.407050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.407052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.407125 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.407447 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.407451 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.407543 (+ 92us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.407625 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.407762 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260928 06:40:36.408972 14915 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.151.252:43149: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0"
member_type: NON_VOTER
last_known_addr {
host: "127.8.151.252"
port: 43149
}
attrs {
promote: true
}
}
dest_uuid: "af8188b892fd4053a8fc54d272088786"
cas_config_opid_index: 5
I20260928 06:40:36.409490 14915 consensus_queue.cc:237] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } }
I20260928 06:40:36.410408 15474 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6a5be8b6352846d5a16b1c6a50b155d0 (127.8.151.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.411929 15474 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.151.252:43149
I20260928 06:40:36.412089 15474 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.412177 15474 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:36.413149 15607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:36.414474 15607 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 FOLLOWER]: Refusing update from remote peer af8188b892fd4053a8fc54d272088786: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260928 06:40:36.415093 15473 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:36.415349 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.410606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:44903 (local address 127.0.0.1:60034)
0928 06:40:36.411012 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.411024 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.411124 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.411544 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.411546 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.411563 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:36.411635 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.411642 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.412680 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.412684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.413885 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.413896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.414945 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.414948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.415047 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.415087 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.415176 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.415243 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343}
I20260928 06:40:36.415719 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.410821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.8.151.253:44903)
0928 06:40:36.410931 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.410935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.411070 (+ 135us) server_negotiation.cc:407] Connection header received
0928 06:40:36.411266 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.411270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.411326 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.411386 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:36.411759 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.412512 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.414021 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.414635 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.414710 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.414854 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.415537 (+ 683us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.415610 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 06:40:36.416144 15525 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 LEARNER]: Refusing update from remote peer af8188b892fd4053a8fc54d272088786: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260928 06:40:36.416461 15496 master.cc:537] Master@127.8.151.252:43149 shutting down...
I20260928 06:40:36.416536 15473 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260928 06:40:36.418413 15525 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6a5be8b6352846d5a16b1c6a50b155d0 (127.8.151.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.420624 15661 sys_catalog.cc:455] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.420729 15661 sys_catalog.cc:458] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: This master's current role is: LEARNER
I20260928 06:40:36.421394 15607 raft_consensus.cc:858] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } }
W20260928 06:40:36.421487 15607 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } }
I20260928 06:40:36.422824 15662 mvcc.cc:205] Tried to move back new op lower bound from 7334205993376256000 to 7334205993344970752. Current Snapshot: MvccSnapshot[applied={T|T < 7334205993376256000}]
I20260928 06:40:36.423327 15554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af8188b892fd4053a8fc54d272088786. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.423410 15554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: This master's current role is: LEARNER
I20260928 06:40:36.424994 15607 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f177e7daf9234317ba18e3de53eef1b7 (127.8.151.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } }
I20260928 06:40:36.426720 15639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af8188b892fd4053a8fc54d272088786. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } }
I20260928 06:40:36.426818 15639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260928 06:40:36.426901 15639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } }
I20260928 06:40:36.426977 15639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260928 06:40:36.427033 15639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } }
I20260928 06:40:36.427067 15639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260928 06:40:36.429141 15473 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: NON_VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: true } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.429251 15473 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:36.430002 15496 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260928 06:40:36.430243 15496 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260928 06:40:36.430320 15496 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: stopping tablet replica
W20260928 06:40:36.430500 14900 consensus_peers.cc:487] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 -> Peer 6a5be8b6352846d5a16b1c6a50b155d0 (127.8.151.252:43149): Couldn't send request to peer 6a5be8b6352846d5a16b1c6a50b155d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 06:40:36.430893 15474 raft_consensus.cc:1050] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786: attempting to promote NON_VOTER f177e7daf9234317ba18e3de53eef1b7 to VOTER
I20260928 06:40:36.431290 15474 consensus_queue.cc:237] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } }
I20260928 06:40:36.432456 15525 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 LEARNER]: Refusing update from remote peer af8188b892fd4053a8fc54d272088786: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260928 06:40:36.433247 15473 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260928 06:40:36.433588 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.433290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:43149 (local address 127.0.0.1:48406)
0928 06:40:36.433426 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.433512 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:43149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260928 06:40:36.434329 14900 consensus_peers.cc:487] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 -> Peer 6a5be8b6352846d5a16b1c6a50b155d0 (127.8.151.252:43149): Couldn't send request to peer 6a5be8b6352846d5a16b1c6a50b155d0. Status: Network error: Client connection negotiation failed: client connection to 127.8.151.252:43149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 06:40:36.435751 15474 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f177e7daf9234317ba18e3de53eef1b7 (127.8.151.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.436887 15474 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.437000 15474 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:36.437120 15474 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.437181 15474 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:36.437634 15525 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f177e7daf9234317ba18e3de53eef1b7 (127.8.151.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.438766 15554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.438854 15554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:40:36.446511 15496 master.cc:552] Master@127.8.151.252:43149 shutdown complete.
I20260928 06:40:36.446559 15496 master_runner.cc:305] Clearing existing system tablet
I20260928 06:40:36.446779 15496 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:36.446853 15496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:36.447995 15496 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:36.448725 15671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.448916 15496 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:36.449034 15496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:36.449070 15496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal
uuid: "6a5be8b6352846d5a16b1c6a50b155d0"
format_stamp: "Formatted at 2026-09-28 06:40:36 on dist-test-slave-gtr8"
server_key: "ceeb62d85cbf627e3ee48183d65b8e9e"
server_key_iv: "807be7da8b68ad6b74f6dd8ec4093c8b"
server_key_version: "encryptionkey@0"
I20260928 06:40:36.450448 15496 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:36.452394 15496 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 06:40:36.452469 15496 log.cc:1196] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260928 06:40:36.452701 15496 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: Deleting consensus metadata
I20260928 06:40:36.452932 15496 master_runner.cc:315] Copying system tablet from 127.8.151.254:33147
I20260928 06:40:36.454049 15496 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: tablet copy: Beginning tablet copy session from remote peer at address 127.8.151.254:33147
I20260928 06:40:36.466775 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.459571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:45764)
0928 06:40:36.459944 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.459956 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.460080 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.460384 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.460387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.460594 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.460684 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.460691 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.461471 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.461475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.462133 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.462146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.462609 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.462613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.462701 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.463267 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.463281 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.463606 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.465291 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.465294 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.465297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.465500 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.465768 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.465771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.465773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.465858 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.466236 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.466240 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.466337 (+ 97us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.466434 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.466581 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":106,"threads_started":1}
I20260928 06:40:36.466774 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.459655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.8.151.254:33147)
0928 06:40:36.459814 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.459820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.460042 (+ 222us) server_negotiation.cc:407] Connection header received
0928 06:40:36.460175 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.460179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.460269 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.460351 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.460818 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.461358 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.462262 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.462490 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.462583 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.463764 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.463767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.463769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.463794 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.465174 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.465596 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.465600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.465602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.465682 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.465943 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.465947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.465950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.466135 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.466229 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.466497 (+ 268us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.466605 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 06:40:36.467549 14935 tablet_copy_service.cc:140] P af8188b892fd4053a8fc54d272088786: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6a5be8b6352846d5a16b1c6a50b155d0 ({username='oryx', principal='oryx/127.8.151.252@KRBTEST.COM'} at 127.0.0.1:45764)
I20260928 06:40:36.467648 14935 tablet_copy_service.cc:161] P af8188b892fd4053a8fc54d272088786: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6a5be8b6352846d5a16b1c6a50b155d0 at {username='oryx', principal='oryx/127.8.151.252@KRBTEST.COM'} at 127.0.0.1:45764: session id = 6a5be8b6352846d5a16b1c6a50b155d0-00000000000000000000000000000000
I20260928 06:40:36.468581 14935 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 06:40:36.469733 15496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 06:40:36.471655 15496 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: tablet copy: Starting download of 0 data blocks...
I20260928 06:40:36.471853 15496 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: tablet copy: Starting download of 1 WAL segments...
I20260928 06:40:36.473186 15496 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 06:40:36.474647 14935 tablet_copy_service.cc:342] P af8188b892fd4053a8fc54d272088786: Request end of tablet copy session 6a5be8b6352846d5a16b1c6a50b155d0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.151.252@KRBTEST.COM'} at 127.0.0.1:45764
I20260928 06:40:36.474726 14935 tablet_copy_service.cc:433] P af8188b892fd4053a8fc54d272088786: ending tablet copy session 6a5be8b6352846d5a16b1c6a50b155d0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6a5be8b6352846d5a16b1c6a50b155d0
I20260928 06:40:36.481402 15496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:36.483011 15681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:36.483098 15678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:36.487234 15679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:36.487319 15496 server_base.cc:1034] running on GCE node
I20260928 06:40:36.487596 15496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:36.487884 15496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:36.489017 15496 hybrid_clock.cc:648] HybridClock initialized: now 1790577636489007 us; error 35 us; skew 500 ppm
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:36.490388 15496 init.cc:377] Logged in from keytab as oryx/127.8.151.252@KRBTEST.COM (short username oryx)
I20260928 06:40:36.490810 15496 webserver.cc:460] Webserver started at http://127.8.151.252:34311/ using document root <none> and password file <none>
I20260928 06:40:36.490988 15496 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:36.491022 15496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:36.492080 15496 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:36.492902 15688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.493191 15496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:36.493254 15496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal
uuid: "6a5be8b6352846d5a16b1c6a50b155d0"
format_stamp: "Formatted at 2026-09-28 06:40:36 on dist-test-slave-gtr8"
server_key: "ceeb62d85cbf627e3ee48183d65b8e9e"
server_key_iv: "807be7da8b68ad6b74f6dd8ec4093c8b"
server_key_version: "encryptionkey@0"
I20260928 06:40:36.493343 15496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:36.501243 15496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:36.503131 15496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:36.508045 15496 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.252:43149
I20260928 06:40:36.508064 15740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.252:43149 every 8 connection(s)
I20260928 06:40:36.508605 15496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
I20260928 06:40:36.510591 15741 sys_catalog.cc:263] Verifying existing consensus state
I20260928 06:40:36.511569 15741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: Bootstrap starting.
I20260928 06:40:36.519166 15741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:40:36.519454 15741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: Bootstrap complete.
I20260928 06:40:36.519975 15741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } }
I20260928 06:40:36.520092 15741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a5be8b6352846d5a16b1c6a50b155d0, State: Initialized, Role: LEARNER
I20260928 06:40:36.520201 15741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } }
I20260928 06:40:36.520541 15744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.520699 15741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:36.520723 15744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: This master's current role is: LEARNER
I20260928 06:40:36.524328 15755 catalog_manager.cc:1259] Loaded cluster ID: 9239a86ffe28446c88565c487ee3f0f2
I20260928 06:40:36.524381 15755 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: loading cluster ID for follower catalog manager: success
I20260928 06:40:36.526109 15755 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: acquiring CA information for follower catalog manager: success
I20260928 06:40:36.527293 15755 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260928 06:40:36.531885 15659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:36.532192 15659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:36.532253 15659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:36.538142 15659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:36.538213 15659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:36.538292 15659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:36.544385 15659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab
--principal=oryx/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:35507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=32847
--tserver_master_addrs=127.8.151.254:33147,127.8.151.253:44903,127.8.151.252:43149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:36.545349 15659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:36.546283 15659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:36.555896 15763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:36.555914 15760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:36.555980 15659 server_base.cc:1034] running on GCE node
W20260928 06:40:36.555943 15761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:36.556452 15659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:36.557042 15659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:36.558188 15659 hybrid_clock.cc:648] HybridClock initialized: now 1790577636558170 us; error 29 us; skew 500 ppm
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:36.560683 15659 init.cc:377] Logged in from keytab as oryx/127.8.151.193@KRBTEST.COM (short username oryx)
I20260928 06:40:36.561497 15659 webserver.cc:460] Webserver started at http://127.8.151.193:32847/ using document root <none> and password file <none>
I20260928 06:40:36.562006 15659 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:36.562085 15659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:36.565824 15659 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:36.567806 15770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.568656 15240 heartbeater.cc:503] Master 127.8.151.254:33147 was elected leader, sending a full tablet report...
I20260928 06:40:36.569183 15659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:36.569298 15659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "c9ec552b32f84d568fa60fea398a060c"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "7c10de6842fa47e54e1a57a0ec041b08"
server_key_iv: "58cf85879f76ec9eef082583f07bcfeb"
server_key_version: "encryptionkey@0"
I20260928 06:40:36.569868 15659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:36.606573 15659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:36.609373 15659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:36.609531 15659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:36.610064 15659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:36.610867 15659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:36.610910 15659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.610948 15659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:36.610966 15659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.619637 15659 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:35507
I20260928 06:40:36.619736 15883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:35507 every 8 connection(s)
I20260928 06:40:36.620620 15659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:36.626641 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 15659
I20260928 06:40:36.626808 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 15111
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:36.632840 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:39851
--local_ip_for_outbound_sockets=127.8.151.194
--tserver_master_addrs=127.8.151.254:33147
--webserver_port=41337
--webserver_interface=127.8.151.194
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab
--principal=oryx/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.151.254:33147,127.8.151.253:44903
--tserver_master_addrs=127.8.151.254:33147,127.8.151.253:44903,127.8.151.252:43149 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.252@KRBTEST.COM
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.253@KRBTEST.COM
I20260928 06:40:36.638396 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.622392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:34889 (local address 127.8.151.254:33147)
0928 06:40:36.622543 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.622548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.623468 (+ 920us) server_negotiation.cc:407] Connection header received
0928 06:40:36.624601 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.624605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.624656 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.624706 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.627421 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.628217 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.630003 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.630390 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.630494 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.634450 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.634453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.634455 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.634480 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.636637 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.637127 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.637131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.637133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.637219 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.637528 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.637531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.637532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.637740 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.637809 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.638115 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.638231 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:40:36.638876 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.622321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:42625 (local address 127.8.151.253:44903)
0928 06:40:36.623290 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.623295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.623476 (+ 181us) server_negotiation.cc:407] Connection header received
0928 06:40:36.624601 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.624605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.624659 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.624706 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.626045 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.626825 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.633257 (+ 6432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.633533 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.633593 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.635750 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.635754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.635757 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.635785 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.637256 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.637666 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.637670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.637672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.637747 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.638038 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.638042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.638044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.638237 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.638314 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.638635 (+ 321us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.638741 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882}
I20260928 06:40:36.639654 15890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.623853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:43149 (local address 127.8.151.193:48243)
0928 06:40:36.624084 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.624098 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.624381 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.624870 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.624878 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.625308 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.625883 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.625898 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.628687 (+ 2789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.628691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.629872 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.629882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.631127 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.631131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.631250 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.631667 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.631681 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.633599 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.636105 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.636111 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.636127 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.636433 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.636754 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.636757 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.636758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.636947 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.637420 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.637428 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.637748 (+ 320us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.637949 (+ 201us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.638290 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":103,"spinlock_wait_cycles":4352,"thread_start_us":69,"threads_started":1}
I20260928 06:40:36.639653 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.622784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:44903 (local address 127.8.151.193:42625)
0928 06:40:36.623308 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.623367 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.624380 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.624871 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.624879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.625309 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.625891 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.625900 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.632437 (+ 6537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.632441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.633125 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.633137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.633740 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.633744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.633826 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.634231 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.634245 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.635594 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.637350 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.637353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.637354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.637548 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.637845 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.637848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.637849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.637925 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.638367 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.638369 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.638463 (+ 94us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.639064 (+ 601us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.639214 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":52,"spinlock_wait_cycles":134528,"thread_start_us":137,"threads_started":1}
I20260928 06:40:36.639889 15889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.622785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.193:34889)
0928 06:40:36.623308 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.623367 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.624381 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.624871 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.624879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.625309 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.625893 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.625900 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.628367 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.628371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.629872 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.629882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.630536 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.630539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.630685 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.631486 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.631506 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.634323 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.636772 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.636775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.636777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.637012 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.637323 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.637326 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.637328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.637411 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.637868 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.637870 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.637950 (+ 80us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.638562 (+ 612us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.638690 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":174,"spinlock_wait_cycles":93440,"thread_start_us":118,"threads_started":1}
I20260928 06:40:36.640990 15891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.623951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:48243 (local address 127.8.151.252:43149)
0928 06:40:36.624293 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.624302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.624313 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:40:36.624572 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.624575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.624630 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.624705 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.626021 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.628564 (+ 2543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.630621 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.631003 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.631097 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.633775 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.633778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.633781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.633809 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.635982 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.636554 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.636557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.636559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.636645 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.637065 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.637068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.637070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.637294 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.640569 (+ 3275us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.640684 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.640814 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":97,"threads_started":1}
I20260928 06:40:36.641258 15886 heartbeater.cc:347] Connected to a master server at 127.8.151.253:44903
I20260928 06:40:36.641603 15884 heartbeater.cc:347] Connected to a master server at 127.8.151.252:43149
I20260928 06:40:36.643002 15884 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:36.643420 15885 heartbeater.cc:347] Connected to a master server at 127.8.151.254:33147
I20260928 06:40:36.643509 15885 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:36.643584 15884 heartbeater.cc:511] Master 127.8.151.252:43149 requested a full tablet report, sending...
I20260928 06:40:36.643638 15885 heartbeater.cc:511] Master 127.8.151.254:33147 requested a full tablet report, sending...
I20260928 06:40:36.644111 15886 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:36.644263 15886 heartbeater.cc:511] Master 127.8.151.253:44903 requested a full tablet report, sending...
I20260928 06:40:36.645205 15705 ts_manager.cc:194] Registered new tserver with Master: c9ec552b32f84d568fa60fea398a060c (127.8.151.193:35507)
I20260928 06:40:36.645205 15515 ts_manager.cc:194] Registered new tserver with Master: c9ec552b32f84d568fa60fea398a060c (127.8.151.193:35507)
I20260928 06:40:36.645911 14915 ts_manager.cc:194] Re-registered known tserver with Master: c9ec552b32f84d568fa60fea398a060c (127.8.151.193:35507)
I20260928 06:40:36.646639 14915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:34889
W20260928 06:40:36.742954 15892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:36.743242 15892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:36.743288 15892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:36.747244 15892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:36.747296 15892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:36.747357 15892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:36.752586 15892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab
--principal=oryx/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:39851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=41337
--tserver_master_addrs=127.8.151.254:33147,127.8.151.253:44903,127.8.151.252:43149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:36.753582 15892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:36.754717 15892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:36.763795 15900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:36.763862 15892 server_base.cc:1034] running on GCE node
W20260928 06:40:36.763806 15897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:36.763803 15898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:36.764389 15892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:36.765005 15892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:36.766147 15892 hybrid_clock.cc:648] HybridClock initialized: now 1790577636766120 us; error 39 us; skew 500 ppm
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:36.768793 15892 init.cc:377] Logged in from keytab as oryx/127.8.151.194@KRBTEST.COM (short username oryx)
I20260928 06:40:36.769618 15892 webserver.cc:460] Webserver started at http://127.8.151.194:41337/ using document root <none> and password file <none>
I20260928 06:40:36.770131 15892 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:36.770207 15892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:36.774107 15892 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:40:36.776186 15374 heartbeater.cc:503] Master 127.8.151.254:33147 was elected leader, sending a full tablet report...
I20260928 06:40:36.776211 15907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.777652 15892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:40:36.777750 15892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "2810e72ad0674063943bde466e9dc35f"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "d18d71d07a2e4b305afbb446d3d53ee2"
server_key_iv: "b3af687c0bdab90eb50407e104fafb05"
server_key_version: "encryptionkey@0"
I20260928 06:40:36.778271 15892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:36.789384 15892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:36.792248 15892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:36.792410 15892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:36.793002 15892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:36.793843 15892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:36.793890 15892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.793942 15892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:36.793970 15892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.802158 15892 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:39851
I20260928 06:40:36.802189 16020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:39851 every 8 connection(s)
I20260928 06:40:36.803079 15892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:36.809374 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 15892
I20260928 06:40:36.809546 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 15245
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.252@KRBTEST.COM
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.253@KRBTEST.COM
I20260928 06:40:36.817399 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:46477
--local_ip_for_outbound_sockets=127.8.151.195
--tserver_master_addrs=127.8.151.254:33147
--webserver_port=41049
--webserver_interface=127.8.151.195
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab
--principal=oryx/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.151.254:33147,127.8.151.253:44903
--tserver_master_addrs=127.8.151.254:33147,127.8.151.253:44903,127.8.151.252:43149 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:40:36.818799 15891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.804739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:52585 (local address 127.8.151.252:43149)
0928 06:40:36.804870 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.804875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.805753 (+ 878us) server_negotiation.cc:407] Connection header received
0928 06:40:36.806843 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.806847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.806905 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.806955 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.808345 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.809160 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.811201 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.811534 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.811613 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.814487 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.814491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.814494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.814524 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.816416 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.817013 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.817017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.817019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.817102 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.817692 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.817695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.817697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.817901 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.817956 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.818508 (+ 552us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.818640 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:40:36.819065 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.806217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:38963 (local address 127.8.151.254:33147)
0928 06:40:36.806357 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.806362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.806381 (+ 19us) server_negotiation.cc:407] Connection header received
0928 06:40:36.807074 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.807078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.807127 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.807218 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.808356 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.808881 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.810599 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.810902 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.810975 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.815017 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.815021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.815024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.815052 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.816779 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.817229 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.817232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.817234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.817317 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.817622 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.817625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.817627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.817817 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.817879 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.818790 (+ 911us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.818909 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 06:40:36.820118 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.804736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:35513 (local address 127.8.151.253:44903)
0928 06:40:36.804859 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.804864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.805775 (+ 911us) server_negotiation.cc:407] Connection header received
0928 06:40:36.806849 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.806853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.806902 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.806975 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:36.808391 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.809165 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.810617 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.810930 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.810998 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.815163 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:36.815167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:36.815170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:36.815198 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:36.816640 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.817096 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.817100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.817101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.817215 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:36.817495 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:36.817498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:36.817500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:36.817715 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:36.817783 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.819855 (+ 2072us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.819969 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:40:36.820695 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.805154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:44903 (local address 127.8.151.194:35513)
0928 06:40:36.805648 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.805662 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.806634 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.807115 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.807121 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.807663 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.808206 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.808222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.809287 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.809290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.810492 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.810503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.811184 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.811188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.811418 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.812296 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.812320 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.814973 (+ 2653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.816741 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.816744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.816746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.817004 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.817315 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.817318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.817320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.817403 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.819592 (+ 2189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.819595 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.819677 (+ 82us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.820304 (+ 627us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.820440 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":441,"spinlock_wait_cycles":40320,"thread_start_us":111,"threads_started":1}
I20260928 06:40:36.820698 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.806103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.194:38963)
0928 06:40:36.806328 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.806338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.806633 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.807230 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.807233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.807661 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.808206 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.808222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.809017 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.809019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.810473 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.810483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.811183 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.811187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.811452 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.812048 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.812067 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.814848 (+ 2781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.816879 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.816882 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.816884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.817117 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.817425 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.817428 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.817430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.817526 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.817954 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.817960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.818271 (+ 311us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.819239 (+ 968us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.819603 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":133,"spinlock_wait_cycles":12928,"thread_start_us":70,"threads_started":1}
I20260928 06:40:36.821451 16025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.805154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:43149 (local address 127.8.151.194:52585)
0928 06:40:36.805608 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.805657 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.806627 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.807116 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.807123 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.807652 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:36.808205 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.808219 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.809287 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.809290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.811060 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.811069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.811779 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.811782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.811904 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.812296 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:36.812318 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:36.814331 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:36.816543 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.816549 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.816566 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.816902 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.817209 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:36.817211 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:36.817213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:36.817388 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:36.818013 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:36.818015 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:36.818270 (+ 255us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.819368 (+ 1098us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.819681 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":1100,"spinlock_wait_cycles":23552,"thread_start_us":111,"threads_started":1}
I20260928 06:40:36.821973 16023 heartbeater.cc:347] Connected to a master server at 127.8.151.253:44903
I20260928 06:40:36.821979 16022 heartbeater.cc:347] Connected to a master server at 127.8.151.254:33147
I20260928 06:40:36.822196 16022 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:36.822504 16021 heartbeater.cc:347] Connected to a master server at 127.8.151.252:43149
I20260928 06:40:36.822566 16021 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:36.822762 16021 heartbeater.cc:511] Master 127.8.151.252:43149 requested a full tablet report, sending...
I20260928 06:40:36.822780 16022 heartbeater.cc:511] Master 127.8.151.254:33147 requested a full tablet report, sending...
I20260928 06:40:36.822196 16023 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:36.823012 16023 heartbeater.cc:511] Master 127.8.151.253:44903 requested a full tablet report, sending...
I20260928 06:40:36.823864 14915 ts_manager.cc:194] Re-registered known tserver with Master: 2810e72ad0674063943bde466e9dc35f (127.8.151.194:39851)
I20260928 06:40:36.823880 15705 ts_manager.cc:194] Registered new tserver with Master: 2810e72ad0674063943bde466e9dc35f (127.8.151.194:39851)
I20260928 06:40:36.823880 15515 ts_manager.cc:194] Registered new tserver with Master: 2810e72ad0674063943bde466e9dc35f (127.8.151.194:39851)
I20260928 06:40:36.824599 14915 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:38963
W20260928 06:40:36.928213 16028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:36.928455 16028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:36.928493 16028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:36.932488 16028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:36.932543 16028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:36.932600 16028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:36.937780 16028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab
--principal=oryx/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:46477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=41049
--tserver_master_addrs=127.8.151.254:33147,127.8.151.253:44903,127.8.151.252:43149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:36.938723 16028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:36.939731 16028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:40:36.949072 16028 server_base.cc:1034] running on GCE node
W20260928 06:40:36.949028 16033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:36.949106 16034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:36.949028 16036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:36.949602 16028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:36.950209 16028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:36.951359 16028 hybrid_clock.cc:648] HybridClock initialized: now 1790577636951339 us; error 32 us; skew 500 ppm
Sep 28 06:40:36 dist-test-slave-gtr8 krb5kdc[14870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:36.954030 16028 init.cc:377] Logged in from keytab as oryx/127.8.151.195@KRBTEST.COM (short username oryx)
I20260928 06:40:36.954845 16028 webserver.cc:460] Webserver started at http://127.8.151.195:41049/ using document root <none> and password file <none>
I20260928 06:40:36.955454 16028 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:36.955529 16028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:36.959093 16028 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:36.961091 16043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.962247 16028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:36.962342 16028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "b4c77749c17942f3bece23898e90320c"
format_stamp: "Formatted at 2026-09-28 06:40:35 on dist-test-slave-gtr8"
server_key: "2ea32a0289152db1da98ff366c739b76"
server_key_iv: "a9dfa29e1c1e1b12de7f440e187b2b31"
server_key_version: "encryptionkey@0"
I20260928 06:40:36.962843 16028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:36.982012 15891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.978167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.8.151.252:43149)
0928 06:40:36.978293 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:36.978298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:36.978441 (+ 143us) server_negotiation.cc:407] Connection header received
0928 06:40:36.978565 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:36.978567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:36.978623 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:36.978717 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:36.978931 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.979488 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.980924 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.981407 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.981535 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.981736 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:36.981794 (+ 58us) server_negotiation.cc:299] Negotiation successful
0928 06:40:36.981891 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 06:40:36.982048 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:36.978070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:43149 (local address 127.0.0.1:46050)
0928 06:40:36.978328 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:36.978341 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:36.978469 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:36.978731 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:36.978733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:36.978746 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:36.978847 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.978853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.979580 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.979583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.980791 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:36.980800 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:36.981561 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:36.981565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:36.981663 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:36.981690 (+ 27us) client_negotiation.cc:769] Sending connection context
0928 06:40:36.981767 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 06:40:36.981853 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":77,"threads_started":1}
I20260928 06:40:36.982594 15715 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260928 06:40:36.982851 15744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af8188b892fd4053a8fc54d272088786. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:36.982990 15744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: This master's current role is: LEARNER
I20260928 06:40:36.987880 16028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:36.990483 16028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:36.990648 16028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:36.991223 16028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:36.992033 16028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:36.992077 16028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:36.992127 16028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:36.992154 16028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:37.000291 16028 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:46477
I20260928 06:40:37.000337 16157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:46477 every 8 connection(s)
I20260928 06:40:37.001245 16028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:37.005007 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 16028
Sep 28 06:40:37 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
Sep 28 06:40:37 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.253@KRBTEST.COM
I20260928 06:40:37.018061 16171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.007471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.8.151.254:33147)
0928 06:40:37.009019 (+ 1548us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.009023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.009036 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:37.009070 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.009073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.009115 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.009188 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:37.010012 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.010740 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.011854 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.012207 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.012291 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.013968 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:37.013972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:37.013975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:37.014001 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:37.015986 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.016458 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.016462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.016464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.016559 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.016849 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.016852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.016854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.017085 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:37.017208 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.017743 (+ 535us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.017863 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1476,"thread_start_us":71,"threads_started":1}
Sep 28 06:40:37 dist-test-slave-gtr8 krb5kdc[14870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577636, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.252@KRBTEST.COM
I20260928 06:40:37.019976 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.003001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:45825 (local address 127.8.151.254:33147)
0928 06:40:37.003130 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.003158 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.004269 (+ 1111us) server_negotiation.cc:407] Connection header received
0928 06:40:37.005615 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.005618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.005673 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.005723 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:37.007772 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.008535 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.010139 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.010486 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.010586 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.015444 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:37.015447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:37.015450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:37.015474 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:37.017569 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.018135 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.018138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.018140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.018217 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.018563 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.018566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.018567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.018741 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:37.018817 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.019707 (+ 890us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.019822 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:40:37.020640 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.003006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:60289 (local address 127.8.151.253:44903)
0928 06:40:37.003130 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.003164 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.004262 (+ 1098us) server_negotiation.cc:407] Connection header received
0928 06:40:37.005615 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.005618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.005673 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.005723 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:37.007666 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.008455 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.010861 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.011216 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.011305 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.014539 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:37.014543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:37.014546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:37.014571 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:37.017206 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.017817 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.017821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.017823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.017906 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.018555 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.018558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.018559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.018764 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:37.018870 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.020347 (+ 1477us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.020466 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":53}
I20260928 06:40:37.021875 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.008088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.8.151.252:43149)
0928 06:40:37.008545 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.008556 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.009461 (+ 905us) server_negotiation.cc:407] Connection header received
0928 06:40:37.009624 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.009628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.009676 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.009732 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:37.010563 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.011307 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.012367 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.012695 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.012765 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.014220 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:37.014224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:37.014227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:37.014252 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:37.016160 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.020546 (+ 4386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.020549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.020551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.020642 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.020932 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.020935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.020937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.021168 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:37.021509 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.021579 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.021726 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":67,"threads_started":1}
I20260928 06:40:37.022531 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.003559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:44903 (local address 127.8.151.195:60289)
0928 06:40:37.004096 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:37.004155 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:37.005372 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:37.006115 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:37.006123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:37.006843 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:37.007564 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.007574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.008573 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.008576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.010753 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.010762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.011430 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.011433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.011867 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.012352 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:37.012367 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:37.014347 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:37.017318 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:37.017325 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:37.017344 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:37.017695 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:37.018003 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:37.018006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:37.018008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:37.018445 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:37.018913 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:37.018916 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:37.019226 (+ 310us) client_negotiation.cc:769] Sending connection context
0928 06:40:37.019429 (+ 203us) client_negotiation.cc:240] Negotiation successful
0928 06:40:37.019814 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":1348,"spinlock_wait_cycles":58624,"thread_start_us":132,"threads_started":1}
I20260928 06:40:37.023444 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.003559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.195:45825)
0928 06:40:37.004096 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:37.004155 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:37.005371 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:37.006115 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:37.006123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:37.006659 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:37.007669 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.007674 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.008643 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.008646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.010017 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.010026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.010976 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.010979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.011212 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.011827 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:37.011847 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:37.015301 (+ 3454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:37.017788 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:37.017791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:37.017793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:37.018024 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:37.018307 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:37.018310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:37.018312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:37.018461 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:37.018870 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:37.018876 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:37.019224 (+ 348us) client_negotiation.cc:769] Sending connection context
0928 06:40:37.019430 (+ 206us) client_negotiation.cc:240] Negotiation successful
0928 06:40:37.021215 (+ 1785us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":2220,"spinlock_wait_cycles":29440,"thread_start_us":132,"threads_started":1}
I20260928 06:40:37.020991 14915 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 06:40:37.024609 16159 heartbeater.cc:347] Connected to a master server at 127.8.151.254:33147
I20260928 06:40:37.024649 16160 heartbeater.cc:347] Connected to a master server at 127.8.151.253:44903
I20260928 06:40:37.024839 16159 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:37.024838 16160 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:37.025417 15891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.004708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:41605 (local address 127.8.151.252:43149)
0928 06:40:37.004821 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.004825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.004908 (+ 83us) server_negotiation.cc:407] Connection header received
0928 06:40:37.005614 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.005617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.005676 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.005728 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:37.007606 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.008332 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.016975 (+ 8643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.017319 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.017384 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.019739 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:37.019742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:37.019745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:37.019771 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:37.021322 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.023164 (+ 1842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.023167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.023169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.023251 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.023559 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.023562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.023564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.023749 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:37.023834 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.025167 (+ 1333us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.025291 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:40:37.025400 16159 heartbeater.cc:511] Master 127.8.151.254:33147 requested a full tablet report, sending...
W20260928 06:40:37.025411 14915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:37.025857 16164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.004602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:43149 (local address 127.8.151.195:41605)
0928 06:40:37.004828 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:37.004841 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:37.005377 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:37.006115 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:37.006124 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:37.006668 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:37.007444 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.007466 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.008497 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.008500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.016850 (+ 8350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.016859 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.017539 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.017542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.017631 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.018040 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:37.018053 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:37.019394 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:37.022808 (+ 3414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:37.022811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:37.022813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:37.023033 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:37.023374 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:37.023377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:37.023379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:37.023463 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:37.024892 (+ 1429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:37.024895 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:37.024984 (+ 89us) client_negotiation.cc:769] Sending connection context
0928 06:40:37.025579 (+ 595us) client_negotiation.cc:240] Negotiation successful
0928 06:40:37.025721 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":2736,"thread_start_us":72,"threads_started":1}
I20260928 06:40:37.026036 16160 heartbeater.cc:511] Master 127.8.151.253:44903 requested a full tablet report, sending...
I20260928 06:40:37.026502 14914 ts_manager.cc:194] Re-registered known tserver with Master: b4c77749c17942f3bece23898e90320c (127.8.151.195:46477)
I20260928 06:40:37.026548 16158 heartbeater.cc:347] Connected to a master server at 127.8.151.252:43149
I20260928 06:40:37.026616 16158 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:37.026693 15515 ts_manager.cc:194] Registered new tserver with Master: b4c77749c17942f3bece23898e90320c (127.8.151.195:46477)
I20260928 06:40:37.026757 16158 heartbeater.cc:511] Master 127.8.151.252:43149 requested a full tablet report, sending...
I20260928 06:40:37.027170 14914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:45825
I20260928 06:40:37.027385 15705 ts_manager.cc:194] Registered new tserver with Master: b4c77749c17942f3bece23898e90320c (127.8.151.195:46477)
I20260928 06:40:37.028012 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.006866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.8.151.253:44903)
0928 06:40:37.007112 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.007117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.008801 (+ 1684us) server_negotiation.cc:407] Connection header received
0928 06:40:37.009900 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.009904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.009956 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.010018 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:37.010861 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.013937 (+ 3076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.014955 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.015303 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.015394 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.016826 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:37.016830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:37.016833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:37.016856 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:37.023295 (+ 6439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.023759 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.023763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.023764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.023849 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.024250 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.024254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.024255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.024466 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:37.027379 (+ 2913us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.027718 (+ 339us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.027843 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":163,"thread_start_us":69,"threads_started":1}
I20260928 06:40:37.029707 15669 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: NON_VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: true } } }
I20260928 06:40:37.029829 15669 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:37.031404 15669 raft_consensus.cc:1050] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786: attempting to promote NON_VOTER 6a5be8b6352846d5a16b1c6a50b155d0 to VOTER
I20260928 06:40:37.031703 15669 consensus_queue.cc:237] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } }
I20260928 06:40:37.032722 15715 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 LEARNER]: Refusing update from remote peer af8188b892fd4053a8fc54d272088786: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260928 06:40:37.032991 15525 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Refusing update from remote peer af8188b892fd4053a8fc54d272088786: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260928 06:40:37.033188 16176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260928 06:40:37.033318 15669 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260928 06:40:37.035043 16178 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6a5be8b6352846d5a16b1c6a50b155d0 (127.8.151.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:37.036541 16176 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:37.036612 16177 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:37.036659 16176 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:37.036684 16177 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:37.038754 15525 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6a5be8b6352846d5a16b1c6a50b155d0 (127.8.151.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:37.038837 15715 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6a5be8b6352846d5a16b1c6a50b155d0 (127.8.151.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:37.040467 15744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:37.036553 16178 sys_catalog.cc:455] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:37.040606 15744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:40:37.040622 16178 sys_catalog.cc:458] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:37.040846 16186 sys_catalog.cc:455] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af8188b892fd4053a8fc54d272088786" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:37.041000 16186 sys_catalog.cc:458] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:40:37.048996 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.043763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:39851 (local address 127.0.0.1:51796)
0928 06:40:37.043914 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:37.043929 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:37.044005 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:37.044754 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:37.044758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:37.044775 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:37.044878 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.044884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.046346 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.046349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.047525 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.047536 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.048581 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.048585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.048689 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.048713 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 06:40:37.048786 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 06:40:37.048859 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260928 06:40:37.049074 16187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.043904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51796 (local address 127.8.151.194:39851)
0928 06:40:37.044206 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.044218 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.044244 (+ 26us) server_negotiation.cc:407] Connection header received
0928 06:40:37.044324 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.044330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.044585 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.044789 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:37.044974 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.046267 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.047651 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.048466 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.048566 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.048713 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.048814 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.048914 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":110,"threads_started":1}
I20260928 06:40:37.049610 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.044284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:46477 (local address 127.0.0.1:60300)
0928 06:40:37.045090 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:37.045102 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:37.045249 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:37.045895 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:37.045897 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:37.045912 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:37.046020 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.046026 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.047121 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.047124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.048079 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.048089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.049229 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.049233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.049323 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.049346 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:40:37.049410 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 06:40:37.049486 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":105,"threads_started":1}
I20260928 06:40:37.050330 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.044429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.8.151.195:46477)
0928 06:40:37.044746 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.044758 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.045180 (+ 422us) server_negotiation.cc:407] Connection header received
0928 06:40:37.045403 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.045409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.045612 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.045759 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:37.046136 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.047033 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.048211 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.048944 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.049035 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.049168 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.050096 (+ 928us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.050195 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":96,"threads_started":1}
I20260928 06:40:37.050462 16190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.044515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.8.151.193:35507)
0928 06:40:37.045944 (+ 1429us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.045955 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.046167 (+ 212us) server_negotiation.cc:407] Connection header received
0928 06:40:37.046369 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.046374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.046556 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.046701 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:37.046888 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.047891 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.048915 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.049679 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.049748 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.049898 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.050236 (+ 338us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.050326 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1343,"thread_start_us":112,"threads_started":1}
I20260928 06:40:37.050822 16188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.044289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:35507 (local address 127.0.0.1:42920)
0928 06:40:37.046077 (+ 1788us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:37.046089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:37.046232 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:37.046693 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:37.046696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:37.046715 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:37.046797 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.046802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.047986 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.047989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.048787 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.048794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.049991 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.049995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.050089 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.050111 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 06:40:37.050629 (+ 518us) client_negotiation.cc:240] Negotiation successful
0928 06:40:37.050704 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1732,"mutex_wait_us":580,"thread_start_us":76,"threads_started":1}
I20260928 06:40:37.052409 15955 tablet_service.cc:1467] Processing CreateTablet for tablet 8f1e0edda31d4b55bfb05ee4e1aeea58 (DEFAULT_TABLE table=test-table [id=06124009dbcc45ab8c05e3467dcea2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:37.053139 16092 tablet_service.cc:1467] Processing CreateTablet for tablet 8f1e0edda31d4b55bfb05ee4e1aeea58 (DEFAULT_TABLE table=test-table [id=06124009dbcc45ab8c05e3467dcea2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:37.053584 15955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f1e0edda31d4b55bfb05ee4e1aeea58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:37.054147 16092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f1e0edda31d4b55bfb05ee4e1aeea58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:37.056664 15818 tablet_service.cc:1467] Processing CreateTablet for tablet 8f1e0edda31d4b55bfb05ee4e1aeea58 (DEFAULT_TABLE table=test-table [id=06124009dbcc45ab8c05e3467dcea2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:37.057467 15818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f1e0edda31d4b55bfb05ee4e1aeea58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:37.061775 16192 tablet_bootstrap.cc:492] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: Bootstrap starting.
I20260928 06:40:37.063519 16193 tablet_bootstrap.cc:492] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: Bootstrap starting.
I20260928 06:40:37.064311 16192 tablet_bootstrap.cc:654] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:37.064699 16194 tablet_bootstrap.cc:492] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: Bootstrap starting.
I20260928 06:40:37.065186 16192 log.cc:824] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:37.065392 16193 tablet_bootstrap.cc:654] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:37.066107 16193 log.cc:824] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:37.066649 16194 tablet_bootstrap.cc:654] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:37.067369 16192 tablet_bootstrap.cc:492] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: No bootstrap required, opened a new log
I20260928 06:40:37.067371 16194 log.cc:824] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:37.067562 16192 ts_tablet_manager.cc:1397] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:40:37.068297 16193 tablet_bootstrap.cc:492] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: No bootstrap required, opened a new log
I20260928 06:40:37.068537 16193 ts_tablet_manager.cc:1397] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:40:37.068866 16194 tablet_bootstrap.cc:492] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: No bootstrap required, opened a new log
I20260928 06:40:37.069036 16194 ts_tablet_manager.cc:1397] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260928 06:40:37.072733 16194 raft_consensus.cc:348] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } }
I20260928 06:40:37.073000 16194 raft_consensus.cc:374] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:37.073062 16194 raft_consensus.cc:729] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ec552b32f84d568fa60fea398a060c, State: Initialized, Role: FOLLOWER
I20260928 06:40:37.072914 16193 raft_consensus.cc:348] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } }
I20260928 06:40:37.073393 16193 raft_consensus.cc:374] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:37.073477 16193 raft_consensus.cc:729] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2810e72ad0674063943bde466e9dc35f, State: Initialized, Role: FOLLOWER
I20260928 06:40:37.073361 16192 raft_consensus.cc:348] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } }
I20260928 06:40:37.073495 16194 consensus_queue.cc:260] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } }
I20260928 06:40:37.073596 16192 raft_consensus.cc:374] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:37.073643 16192 raft_consensus.cc:729] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c77749c17942f3bece23898e90320c, State: Initialized, Role: FOLLOWER
I20260928 06:40:37.074045 16193 consensus_queue.cc:260] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } }
I20260928 06:40:37.074180 16192 consensus_queue.cc:260] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } }
I20260928 06:40:37.074482 16194 ts_tablet_manager.cc:1428] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260928 06:40:37.075086 16192 ts_tablet_manager.cc:1428] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260928 06:40:37.075098 16193 ts_tablet_manager.cc:1428] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:40:37.077129 16159 heartbeater.cc:503] Master 127.8.151.254:33147 was elected leader, sending a full tablet report...
I20260928 06:40:37.077682 15885 heartbeater.cc:503] Master 127.8.151.254:33147 was elected leader, sending a full tablet report...
I20260928 06:40:37.079386 16022 heartbeater.cc:503] Master 127.8.151.254:33147 was elected leader, sending a full tablet report...
W20260928 06:40:37.122628 15887 tablet.cc:2369] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:37.172422 16198 raft_consensus.cc:482] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:37.172612 16198 raft_consensus.cc:504] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } }
I20260928 06:40:37.173729 16198 leader_election.cc:288] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2810e72ad0674063943bde466e9dc35f (127.8.151.194:39851), b4c77749c17942f3bece23898e90320c (127.8.151.195:46477)
I20260928 06:40:37.177476 16187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.174047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:40623 (local address 127.8.151.194:39851)
0928 06:40:37.174169 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.174175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.174347 (+ 172us) server_negotiation.cc:407] Connection header received
0928 06:40:37.174502 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.174505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.174551 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.174605 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:37.174879 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.175411 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.176570 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.176950 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.177046 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.177099 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.177270 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.177339 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:40:37.177510 16202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.173946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:46477 (local address 127.8.151.193:54813)
0928 06:40:37.174291 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:37.174308 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:37.174414 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:37.174651 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:37.174653 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:37.174668 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:37.174766 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.174770 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.175538 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.175542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.176473 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.176482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.177103 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.177107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.177191 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.177207 (+ 16us) client_negotiation.cc:769] Sending connection context
0928 06:40:37.177277 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 06:40:37.177340 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":118,"thread_start_us":110,"threads_started":1}
I20260928 06:40:37.177501 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.174054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:54813 (local address 127.8.151.195:46477)
0928 06:40:37.174175 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.174179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.174396 (+ 217us) server_negotiation.cc:407] Connection header received
0928 06:40:37.174514 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.174516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.174557 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.174632 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:37.174888 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.175434 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.176590 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.176991 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.177100 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.177148 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.177300 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.177364 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:40:37.177513 16201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.173946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:39851 (local address 127.8.151.193:40623)
0928 06:40:37.174260 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:37.174273 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:37.174367 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:37.174639 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:37.174642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:37.174657 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:37.174775 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.174780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.175514 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.175518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.176455 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:37.176464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.177060 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.177064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:37.177153 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.177177 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 06:40:37.177240 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 06:40:37.177311 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":76,"thread_start_us":70,"threads_started":1}
I20260928 06:40:37.178205 16112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f1e0edda31d4b55bfb05ee4e1aeea58" candidate_uuid: "c9ec552b32f84d568fa60fea398a060c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c77749c17942f3bece23898e90320c" is_pre_election: true
I20260928 06:40:37.178259 15975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f1e0edda31d4b55bfb05ee4e1aeea58" candidate_uuid: "c9ec552b32f84d568fa60fea398a060c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2810e72ad0674063943bde466e9dc35f" is_pre_election: true
I20260928 06:40:37.178490 16112 raft_consensus.cc:2454] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ec552b32f84d568fa60fea398a060c in term 0.
I20260928 06:40:37.178508 15975 raft_consensus.cc:2454] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ec552b32f84d568fa60fea398a060c in term 0.
I20260928 06:40:37.178929 15774 leader_election.cc:302] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4c77749c17942f3bece23898e90320c, c9ec552b32f84d568fa60fea398a060c; no voters:
I20260928 06:40:37.179205 16198 raft_consensus.cc:2781] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:37.179280 16198 raft_consensus.cc:482] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:37.179311 16198 raft_consensus.cc:3037] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:37.180305 16198 raft_consensus.cc:504] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } }
I20260928 06:40:37.180655 16198 leader_election.cc:288] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [CANDIDATE]: Term 1 election: Requested vote from peers 2810e72ad0674063943bde466e9dc35f (127.8.151.194:39851), b4c77749c17942f3bece23898e90320c (127.8.151.195:46477)
I20260928 06:40:37.180979 15975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f1e0edda31d4b55bfb05ee4e1aeea58" candidate_uuid: "c9ec552b32f84d568fa60fea398a060c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2810e72ad0674063943bde466e9dc35f"
I20260928 06:40:37.181022 16112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f1e0edda31d4b55bfb05ee4e1aeea58" candidate_uuid: "c9ec552b32f84d568fa60fea398a060c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c77749c17942f3bece23898e90320c"
I20260928 06:40:37.181092 15975 raft_consensus.cc:3037] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:37.181118 16112 raft_consensus.cc:3037] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:37.182150 15975 raft_consensus.cc:2454] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ec552b32f84d568fa60fea398a060c in term 1.
I20260928 06:40:37.182186 16112 raft_consensus.cc:2454] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ec552b32f84d568fa60fea398a060c in term 1.
I20260928 06:40:37.182410 15773 leader_election.cc:302] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2810e72ad0674063943bde466e9dc35f, c9ec552b32f84d568fa60fea398a060c; no voters:
I20260928 06:40:37.182549 16198 raft_consensus.cc:2781] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:37.182713 16198 raft_consensus.cc:686] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 1 LEADER]: Becoming Leader. State: Replica: c9ec552b32f84d568fa60fea398a060c, State: Running, Role: LEADER
I20260928 06:40:37.183053 16198 consensus_queue.cc:237] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } }
I20260928 06:40:37.186275 14914 catalog_manager.cc:5495] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c reported cstate change: term changed from 0 to 1, leader changed from <none> to c9ec552b32f84d568fa60fea398a060c (127.8.151.193). New cstate: current_term: 1 leader_uuid: "c9ec552b32f84d568fa60fea398a060c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } health_report { overall_health: UNKNOWN } } }
I20260928 06:40:37.205783 16190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.202471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42926 (local address 127.8.151.193:35507)
0928 06:40:37.202601 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.202604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.202619 (+ 15us) server_negotiation.cc:407] Connection header received
0928 06:40:37.202692 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.202694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.202745 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.202831 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:37.203065 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.203631 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.204481 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.204695 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.204770 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.205011 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:37.205467 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:37.205568 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.205599 (+ 31us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.205648 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 06:40:37.211764 16112 raft_consensus.cc:1261] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c [term 1 FOLLOWER]: Refusing update from remote peer c9ec552b32f84d568fa60fea398a060c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:37.211812 15975 raft_consensus.cc:1261] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f [term 1 FOLLOWER]: Refusing update from remote peer c9ec552b32f84d568fa60fea398a060c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:37.212469 16198 consensus_queue.cc:1035] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:37.212667 16203 consensus_queue.cc:1035] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:37.220649 16206 mvcc.cc:205] Tried to move back new op lower bound from 7334206002011623424 to 7334206001904435200. Current Snapshot: MvccSnapshot[applied={T|T < 7334206002011623424}]
I20260928 06:40:37.221689 16207 mvcc.cc:205] Tried to move back new op lower bound from 7334206002011623424 to 7334206001904435200. Current Snapshot: MvccSnapshot[applied={T|T < 7334206002011623424}]
I20260928 06:40:37.223809 16208 mvcc.cc:205] Tried to move back new op lower bound from 7334206002011623424 to 7334206001904435200. Current Snapshot: MvccSnapshot[applied={T|T < 7334206002011623424}]
I20260928 06:40:37.231201 14915 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:37.231352 14915 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:37.236040 14915 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af8188b892fd4053a8fc54d272088786: Sending DeleteTablet for 3 replicas of tablet 8f1e0edda31d4b55bfb05ee4e1aeea58
I20260928 06:40:37.236757 15955 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f1e0edda31d4b55bfb05ee4e1aeea58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:37 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:51796
I20260928 06:40:37.236791 15818 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f1e0edda31d4b55bfb05ee4e1aeea58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:37 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:42920
I20260928 06:40:37.237031 16092 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f1e0edda31d4b55bfb05ee4e1aeea58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:37 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:60300
I20260928 06:40:37.237494 16216 tablet_replica.cc:330] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: stopping tablet replica
I20260928 06:40:37.237823 16216 raft_consensus.cc:2229] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:37.238243 16216 raft_consensus.cc:2258] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:37.239322 16218 tablet_replica.cc:330] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: stopping tablet replica
I20260928 06:40:37.239569 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 14883
I20260928 06:40:37.239707 16218 raft_consensus.cc:2229] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:37.240314 16216 ts_tablet_manager.cc:1905] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:37.240520 16218 raft_consensus.cc:2258] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:37.242129 16218 ts_tablet_manager.cc:1905] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:37.243017 16216 ts_tablet_manager.cc:1918] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:37.243323 16216 log.cc:1196] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/8f1e0edda31d4b55bfb05ee4e1aeea58
I20260928 06:40:37.243649 16216 ts_tablet_manager.cc:1939] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P c9ec552b32f84d568fa60fea398a060c: Deleting consensus metadata
I20260928 06:40:37.244958 16218 ts_tablet_manager.cc:1918] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:37.245107 16218 log.cc:1196] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/8f1e0edda31d4b55bfb05ee4e1aeea58
I20260928 06:40:37.245450 16218 ts_tablet_manager.cc:1939] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P b4c77749c17942f3bece23898e90320c: Deleting consensus metadata
I20260928 06:40:37.245751 16217 tablet_replica.cc:330] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: stopping tablet replica
I20260928 06:40:37.246076 16217 raft_consensus.cc:2229] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:37.246380 16217 raft_consensus.cc:2258] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:37.247879 16217 ts_tablet_manager.cc:1905] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 06:40:37.247988 15773 connection.cc:535] server connection from 127.0.0.1:42920 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260928 06:40:37.248349 16044 connection.cc:535] server connection from 127.0.0.1:60300 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 06:40:37.251317 16217 ts_tablet_manager.cc:1918] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:37.251410 16217 log.cc:1196] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/8f1e0edda31d4b55bfb05ee4e1aeea58
I20260928 06:40:37.251731 16217 ts_tablet_manager.cc:1939] T 8f1e0edda31d4b55bfb05ee4e1aeea58 P 2810e72ad0674063943bde466e9dc35f: Deleting consensus metadata
W20260928 06:40:37.253150 15909 connection.cc:441] server connection from 127.0.0.1:51796 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51796 (request call id 1) could send its response
I20260928 06:40:37.254817 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.238239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53868 (local address 127.8.151.253:44903)
0928 06:40:37.238348 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.238352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.238364 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:37.238399 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.238402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.238445 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.238519 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:37.245720 (+ 7201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.246488 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.248412 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.248773 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.248860 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.251026 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:37.251030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:37.251033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:37.251061 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:37.252865 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.253402 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.253405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.253407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.253502 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.253814 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.253818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.253819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.254052 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:37.254154 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.254519 (+ 365us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.254646 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:40:37.263319 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.254083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46080 (local address 127.8.151.252:43149)
0928 06:40:37.254198 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.254202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.254771 (+ 569us) server_negotiation.cc:407] Connection header received
0928 06:40:37.254871 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.254875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.254932 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.255032 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:37.256112 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.256851 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.258006 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.258329 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.258398 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.260050 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:37.260054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:37.260056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:37.260082 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:37.261561 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.262009 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.262013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.262015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.262090 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.262359 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.262363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.262365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.262562 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:37.262659 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.262985 (+ 326us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.263101 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35,"spinlock_wait_cycles":6400}
I20260928 06:40:37.263792 15891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.238232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.8.151.252:43149)
0928 06:40:37.238332 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.238337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.256204 (+ 17867us) server_negotiation.cc:407] Connection header received
0928 06:40:37.256356 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.256359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.256399 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.256444 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:37.257189 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.257905 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.258879 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.259209 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.259300 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.260672 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:37.260675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:37.260678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:37.260703 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:37.262058 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.262554 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.262557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.262558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.262626 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.262914 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.262917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.262919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.263101 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:37.263219 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.263564 (+ 345us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.263657 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32,"spinlock_wait_cycles":6784}
I20260928 06:40:37.264425 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:37.252130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53878 (local address 127.8.151.253:44903)
0928 06:40:37.252241 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:37.252245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:37.253340 (+ 1095us) server_negotiation.cc:407] Connection header received
0928 06:40:37.253461 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:37.253465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:37.253515 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:37.253582 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:37.255101 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.255906 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.257070 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:37.257403 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:37.257470 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:37.259558 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:37.259562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:37.259564 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:37.259587 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:37.261996 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.262545 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.262549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.262550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.262630 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:37.262947 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:37.262951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:37.262952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:37.263211 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:37.263305 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:37.264172 (+ 867us) server_negotiation.cc:299] Negotiation successful
0928 06:40:37.264278 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:40:38.224511 16227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.224047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.194:55593)
0928 06:40:38.224291 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.224395 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260928 06:40:38.225696 16228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.224397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.193:37253)
0928 06:40:38.225510 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.225600 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1038,"thread_start_us":82,"threads_started":1}
W20260928 06:40:38.226037 15885 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:33147 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
W20260928 06:40:38.226375 16022 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:33147 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
I20260928 06:40:38.226840 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.226097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.195:59177)
0928 06:40:38.226656 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.226735 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":59,"threads_started":1}
W20260928 06:40:38.227133 16159 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:33147 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
I20260928 06:40:38.736301 16230 raft_consensus.cc:482] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader af8188b892fd4053a8fc54d272088786)
I20260928 06:40:38.736466 16231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader af8188b892fd4053a8fc54d272088786)
I20260928 06:40:38.736536 16230 raft_consensus.cc:504] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } }
I20260928 06:40:38.736654 16231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } }
I20260928 06:40:38.737504 16230 leader_election.cc:288] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af8188b892fd4053a8fc54d272088786 (127.8.151.254:33147), 6a5be8b6352846d5a16b1c6a50b155d0 (127.8.151.252:43149)
I20260928 06:40:38.737839 16231 leader_election.cc:288] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af8188b892fd4053a8fc54d272088786 (127.8.151.254:33147), f177e7daf9234317ba18e3de53eef1b7 (127.8.151.253:44903)
I20260928 06:40:38.738117 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.737676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:38064)
0928 06:40:38.737931 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.738008 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":70,"threads_started":1}
W20260928 06:40:38.738298 15690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
I20260928 06:40:38.738581 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.737666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:38062)
0928 06:40:38.738427 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.738492 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":73,"threads_started":1}
W20260928 06:40:38.738907 15500 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
I20260928 06:40:38.739259 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.739025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:38078)
0928 06:40:38.739127 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.739193 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260928 06:40:38.739347 15690 leader_election.cc:334] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer af8188b892fd4053a8fc54d272088786 (127.8.151.254:33147): Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
I20260928 06:40:38.739810 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.739599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:38094)
0928 06:40:38.739694 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.739746 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 06:40:38.739905 15500 leader_election.cc:334] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer af8188b892fd4053a8fc54d272088786 (127.8.151.254:33147): Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
I20260928 06:40:38.742239 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.738126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.8.151.252:43149)
0928 06:40:38.738309 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:38.738314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:38.738685 (+ 371us) server_negotiation.cc:407] Connection header received
0928 06:40:38.738873 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:38.738875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:38.738916 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:38.738995 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:38.739287 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.740050 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.741176 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.741733 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.741812 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.741884 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:38.742076 (+ 192us) server_negotiation.cc:299] Negotiation successful
0928 06:40:38.742136 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":53,"threads_started":1}
I20260928 06:40:38.742245 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.737981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:43149 (local address 127.0.0.1:46084)
0928 06:40:38.738568 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.738581 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:38.738775 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:38.739022 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:38.739025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:38.739043 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:38.739162 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:38.739170 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.740172 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.740176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:38.741069 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:38.741078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.741832 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.741836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:38.741937 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.741962 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 06:40:38.742060 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 06:40:38.742125 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":65,"threads_started":1}
I20260928 06:40:38.742591 16232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.737666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:44903 (local address 127.0.0.1:53880)
0928 06:40:38.737859 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.737886 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:38.738070 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:38.738588 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:38.738592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:38.738613 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:38.738729 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:38.738736 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.740474 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.740477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:38.741443 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:38.741452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.742223 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.742226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:38.742309 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.742343 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 06:40:38.742411 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 06:40:38.742478 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":56,"threads_started":1}
I20260928 06:40:38.742619 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.738144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.8.151.253:44903)
0928 06:40:38.738353 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:38.738358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:38.738370 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:38.738415 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:38.738418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:38.738463 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:38.739244 (+ 781us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:38.739320 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.740385 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.741565 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.742114 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.742196 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.742244 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:38.742450 (+ 206us) server_negotiation.cc:299] Negotiation successful
0928 06:40:38.742518 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":46,"threads_started":1}
I20260928 06:40:38.742713 15715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f177e7daf9234317ba18e3de53eef1b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" is_pre_election: true
I20260928 06:40:38.742909 15715 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f177e7daf9234317ba18e3de53eef1b7 in term 1.
I20260928 06:40:38.743230 15500 leader_election.cc:302] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5be8b6352846d5a16b1c6a50b155d0, f177e7daf9234317ba18e3de53eef1b7; no voters: af8188b892fd4053a8fc54d272088786
I20260928 06:40:38.743296 15525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f177e7daf9234317ba18e3de53eef1b7" is_pre_election: true
I20260928 06:40:38.743479 16230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 06:40:38.743526 16230 raft_consensus.cc:482] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader af8188b892fd4053a8fc54d272088786)
I20260928 06:40:38.743557 16230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 1 FOLLOWER]: Advancing to term 2
I20260928 06:40:38.744547 16230 raft_consensus.cc:504] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } }
I20260928 06:40:38.744769 15525 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5be8b6352846d5a16b1c6a50b155d0 in current term 2: Already voted for candidate f177e7daf9234317ba18e3de53eef1b7 in this term.
I20260928 06:40:38.744903 16230 leader_election.cc:288] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [CANDIDATE]: Term 2 election: Requested vote from peers af8188b892fd4053a8fc54d272088786 (127.8.151.254:33147), 6a5be8b6352846d5a16b1c6a50b155d0 (127.8.151.252:43149)
I20260928 06:40:38.745242 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.745039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:38100)
0928 06:40:38.745134 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.745184 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 06:40:38.745441 15715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f177e7daf9234317ba18e3de53eef1b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6a5be8b6352846d5a16b1c6a50b155d0"
I20260928 06:40:38.745507 15689 leader_election.cc:302] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5be8b6352846d5a16b1c6a50b155d0; no voters: af8188b892fd4053a8fc54d272088786, f177e7daf9234317ba18e3de53eef1b7
I20260928 06:40:38.745584 15715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 1 FOLLOWER]: Advancing to term 2
I20260928 06:40:38.745954 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.745685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:38114)
0928 06:40:38.745823 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.745884 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260928 06:40:38.746073 15500 leader_election.cc:334] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer af8188b892fd4053a8fc54d272088786 (127.8.151.254:33147): Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
I20260928 06:40:38.746670 15715 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f177e7daf9234317ba18e3de53eef1b7 in term 2.
I20260928 06:40:38.746781 16231 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 06:40:38.746995 15500 leader_election.cc:302] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5be8b6352846d5a16b1c6a50b155d0, f177e7daf9234317ba18e3de53eef1b7; no voters: af8188b892fd4053a8fc54d272088786
I20260928 06:40:38.747201 16230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 06:40:38.747428 16230 raft_consensus.cc:686] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [term 2 LEADER]: Becoming Leader. State: Replica: f177e7daf9234317ba18e3de53eef1b7, State: Running, Role: LEADER
I20260928 06:40:38.747694 16230 consensus_queue.cc:237] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } }
I20260928 06:40:38.748800 16240 sys_catalog.cc:455] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f177e7daf9234317ba18e3de53eef1b7. Latest consensus state: current_term: 2 leader_uuid: "f177e7daf9234317ba18e3de53eef1b7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:38.748919 16240 sys_catalog.cc:458] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:38.749202 16242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:38.751425 16242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:38.751788 16242 catalog_manager.cc:1259] Loaded cluster ID: 9239a86ffe28446c88565c487ee3f0f2
I20260928 06:40:38.751833 16242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:38.752349 16242 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:38.752707 16242 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Loaded TSK: 0
I20260928 06:40:38.753330 16242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:40:38.781123 15513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:38.784242 15513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:38.787168 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.786902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:38132)
0928 06:40:38.787010 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.787070 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260928 06:40:38.787462 15715 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [term 2 FOLLOWER]: Refusing update from remote peer f177e7daf9234317ba18e3de53eef1b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260928 06:40:38.787914 16230 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260928 06:40:38.788196 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.787993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.0.0.1:38148)
0928 06:40:38.788096 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.788142 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260928 06:40:38.788368 15500 consensus_peers.cc:487] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 -> Peer af8188b892fd4053a8fc54d272088786 (127.8.151.254:33147): Couldn't send request to peer af8188b892fd4053a8fc54d272088786. Status: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 06:40:38.789273 16231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f177e7daf9234317ba18e3de53eef1b7. Latest consensus state: current_term: 2 leader_uuid: "f177e7daf9234317ba18e3de53eef1b7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:38.789393 16231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:40:38.790401 16230 sys_catalog.cc:455] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f177e7daf9234317ba18e3de53eef1b7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:38.790517 16230 sys_catalog.cc:458] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:38.790920 16231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f177e7daf9234317ba18e3de53eef1b7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "af8188b892fd4053a8fc54d272088786" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 33147 } } peers { permanent_uuid: "f177e7daf9234317ba18e3de53eef1b7" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 44903 } attrs { promote: false } } peers { permanent_uuid: "6a5be8b6352846d5a16b1c6a50b155d0" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 43149 } attrs { promote: false } } }
I20260928 06:40:38.791018 16231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a5be8b6352846d5a16b1c6a50b155d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:40:38.800917 16249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.796181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51812 (local address 127.8.151.194:39851)
0928 06:40:38.796449 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:38.796454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:38.796847 (+ 393us) server_negotiation.cc:407] Connection header received
0928 06:40:38.796961 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:38.796964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:38.797023 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:38.797111 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:38.797389 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.798198 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.799503 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.800201 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.800315 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.800385 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:38.800707 (+ 322us) server_negotiation.cc:299] Negotiation successful
0928 06:40:38.800781 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
I20260928 06:40:38.800916 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.796014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:39851 (local address 127.0.0.1:51812)
0928 06:40:38.796767 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.796781 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:38.796876 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:38.797128 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:38.797132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:38.797150 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:38.797244 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:38.797249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.798298 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.798300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:38.799356 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:38.799367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.800442 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.800446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:38.800573 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.800602 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 06:40:38.800670 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 06:40:38.800746 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":62}
I20260928 06:40:38.800948 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.796276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:46477 (local address 127.0.0.1:60310)
0928 06:40:38.796437 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.796454 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:38.796547 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:38.797319 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:38.797322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:38.797338 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:38.797426 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:38.797433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.798482 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.798485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:38.799415 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:38.799426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.800462 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.800466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:38.800595 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.800621 (+ 26us) client_negotiation.cc:769] Sending connection context
0928 06:40:38.800698 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 06:40:38.800770 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":5}
I20260928 06:40:38.801200 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.796506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60310 (local address 127.8.151.195:46477)
0928 06:40:38.796734 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:38.796738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:38.796752 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:40:38.796802 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:38.796805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:38.796905 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:38.797527 (+ 622us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:38.797631 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.798361 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.799559 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.800285 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.800372 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.800428 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:38.800996 (+ 568us) server_negotiation.cc:299] Negotiation successful
0928 06:40:38.801077 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":75,"threads_started":1}
I20260928 06:40:38.801491 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.796582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:35507 (local address 127.0.0.1:42936)
0928 06:40:38.797374 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.797386 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:38.797461 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:38.797804 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:38.797807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:38.797825 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:38.797924 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:38.797930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.798859 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.798862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:38.799956 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:38.799964 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.801155 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.801159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:38.801239 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.801259 (+ 20us) client_negotiation.cc:769] Sending connection context
0928 06:40:38.801315 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 06:40:38.801375 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":111,"threads_started":1}
I20260928 06:40:38.801718 16092 tablet_service.cc:1467] Processing CreateTablet for tablet bbdd43370f4947018549f060a49a5a85 (DEFAULT_TABLE table=test-table [id=a938d14aaf054ad7bbcd1cf98cd3287c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:38.801870 16252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.796846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.8.151.193:35507)
0928 06:40:38.797102 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:38.797107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:38.797567 (+ 460us) server_negotiation.cc:407] Connection header received
0928 06:40:38.797626 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:38.797629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:38.797676 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:38.797765 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:38.798037 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.798741 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.800106 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.800764 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.800860 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.800932 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:38.801667 (+ 735us) server_negotiation.cc:299] Negotiation successful
0928 06:40:38.801750 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260928 06:40:38.802035 16092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdd43370f4947018549f060a49a5a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:38.802296 15818 tablet_service.cc:1467] Processing CreateTablet for tablet bbdd43370f4947018549f060a49a5a85 (DEFAULT_TABLE table=test-table [id=a938d14aaf054ad7bbcd1cf98cd3287c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:38.802551 15818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdd43370f4947018549f060a49a5a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:38.804800 16253 tablet_bootstrap.cc:492] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c: Bootstrap starting.
I20260928 06:40:38.804991 15955 tablet_service.cc:1467] Processing CreateTablet for tablet bbdd43370f4947018549f060a49a5a85 (DEFAULT_TABLE table=test-table [id=a938d14aaf054ad7bbcd1cf98cd3287c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:38.805172 16254 tablet_bootstrap.cc:492] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c: Bootstrap starting.
I20260928 06:40:38.805241 15955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdd43370f4947018549f060a49a5a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:38.805699 16253 tablet_bootstrap.cc:654] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:38.806020 16254 tablet_bootstrap.cc:654] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:38.806821 16253 tablet_bootstrap.cc:492] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c: No bootstrap required, opened a new log
I20260928 06:40:38.806906 16253 ts_tablet_manager.cc:1397] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:38.807158 16254 tablet_bootstrap.cc:492] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c: No bootstrap required, opened a new log
I20260928 06:40:38.807232 16254 ts_tablet_manager.cc:1397] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:38.807384 16253 raft_consensus.cc:348] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } }
I20260928 06:40:38.807464 16257 tablet_bootstrap.cc:492] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f: Bootstrap starting.
I20260928 06:40:38.807487 16253 raft_consensus.cc:374] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:38.807518 16253 raft_consensus.cc:729] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c77749c17942f3bece23898e90320c, State: Initialized, Role: FOLLOWER
I20260928 06:40:38.807586 16254 raft_consensus.cc:348] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } }
I20260928 06:40:38.807675 16254 raft_consensus.cc:374] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:38.807705 16254 raft_consensus.cc:729] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ec552b32f84d568fa60fea398a060c, State: Initialized, Role: FOLLOWER
I20260928 06:40:38.807664 16253 consensus_queue.cc:260] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } }
I20260928 06:40:38.807821 16254 consensus_queue.cc:260] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } }
I20260928 06:40:38.807963 16253 ts_tablet_manager.cc:1428] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:38.808121 16254 ts_tablet_manager.cc:1428] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:38.808297 16257 tablet_bootstrap.cc:654] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:38.809417 16257 tablet_bootstrap.cc:492] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f: No bootstrap required, opened a new log
I20260928 06:40:38.809504 16257 ts_tablet_manager.cc:1397] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:38.809901 16257 raft_consensus.cc:348] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } }
I20260928 06:40:38.810007 16257 raft_consensus.cc:374] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:38.810034 16257 raft_consensus.cc:729] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2810e72ad0674063943bde466e9dc35f, State: Initialized, Role: FOLLOWER
I20260928 06:40:38.810144 16257 consensus_queue.cc:260] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } }
I20260928 06:40:38.810446 16257 ts_tablet_manager.cc:1428] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:38.811090 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.809187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.193:46969)
0928 06:40:38.810906 (+ 1719us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.810970 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1648,"thread_start_us":100,"threads_started":1}
I20260928 06:40:38.812172 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.811412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.194:44635)
0928 06:40:38.812024 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.812077 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":88,"threads_started":1}
I20260928 06:40:38.812636 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.812396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.193:57025)
0928 06:40:38.812525 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.812573 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260928 06:40:38.812819 15886 heartbeater.cc:503] Master 127.8.151.253:44903 was elected leader, sending a full tablet report...
I20260928 06:40:38.811625 16160 heartbeater.cc:503] Master 127.8.151.253:44903 was elected leader, sending a full tablet report...
I20260928 06:40:38.813174 16023 heartbeater.cc:503] Master 127.8.151.253:44903 was elected leader, sending a full tablet report...
I20260928 06:40:38.813422 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.813030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.194:33991)
0928 06:40:38.813311 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.813362 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260928 06:40:38.811615 16260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.808384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.195:37885)
0928 06:40:38.811460 (+ 3076us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.811517 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3009,"thread_start_us":88,"threads_started":1}
I20260928 06:40:38.815079 16265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.814703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.195:38699)
0928 06:40:38.814942 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.814987 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":100,"threads_started":1}
I20260928 06:40:38.834138 16259 raft_consensus.cc:482] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:38.834285 16259 raft_consensus.cc:504] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } }
I20260928 06:40:38.834748 16259 leader_election.cc:288] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2810e72ad0674063943bde466e9dc35f (127.8.151.194:39851), b4c77749c17942f3bece23898e90320c (127.8.151.195:46477)
I20260928 06:40:38.835198 15975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdd43370f4947018549f060a49a5a85" candidate_uuid: "c9ec552b32f84d568fa60fea398a060c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2810e72ad0674063943bde466e9dc35f" is_pre_election: true
I20260928 06:40:38.835243 16112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdd43370f4947018549f060a49a5a85" candidate_uuid: "c9ec552b32f84d568fa60fea398a060c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c77749c17942f3bece23898e90320c" is_pre_election: true
I20260928 06:40:38.835356 15975 raft_consensus.cc:2454] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ec552b32f84d568fa60fea398a060c in term 0.
I20260928 06:40:38.835400 16112 raft_consensus.cc:2454] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ec552b32f84d568fa60fea398a060c in term 0.
I20260928 06:40:38.835685 15773 leader_election.cc:302] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2810e72ad0674063943bde466e9dc35f, c9ec552b32f84d568fa60fea398a060c; no voters:
I20260928 06:40:38.835844 16259 raft_consensus.cc:2781] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:38.835893 16259 raft_consensus.cc:482] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:38.835918 16259 raft_consensus.cc:3037] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:38.836874 16259 raft_consensus.cc:504] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } }
I20260928 06:40:38.837186 16259 leader_election.cc:288] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [CANDIDATE]: Term 1 election: Requested vote from peers 2810e72ad0674063943bde466e9dc35f (127.8.151.194:39851), b4c77749c17942f3bece23898e90320c (127.8.151.195:46477)
I20260928 06:40:38.837488 15975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdd43370f4947018549f060a49a5a85" candidate_uuid: "c9ec552b32f84d568fa60fea398a060c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2810e72ad0674063943bde466e9dc35f"
I20260928 06:40:38.837510 16112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdd43370f4947018549f060a49a5a85" candidate_uuid: "c9ec552b32f84d568fa60fea398a060c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c77749c17942f3bece23898e90320c"
I20260928 06:40:38.837587 15975 raft_consensus.cc:3037] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:38.837612 16112 raft_consensus.cc:3037] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:38.838439 15975 raft_consensus.cc:2454] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ec552b32f84d568fa60fea398a060c in term 1.
I20260928 06:40:38.838557 16112 raft_consensus.cc:2454] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ec552b32f84d568fa60fea398a060c in term 1.
I20260928 06:40:38.838685 15773 leader_election.cc:302] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2810e72ad0674063943bde466e9dc35f, c9ec552b32f84d568fa60fea398a060c; no voters:
I20260928 06:40:38.838850 16259 raft_consensus.cc:2781] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:38.839082 16259 raft_consensus.cc:686] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [term 1 LEADER]: Becoming Leader. State: Replica: c9ec552b32f84d568fa60fea398a060c, State: Running, Role: LEADER
I20260928 06:40:38.839238 16259 consensus_queue.cc:237] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } }
I20260928 06:40:38.839809 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.839557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.193:46813)
0928 06:40:38.839673 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.839734 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 06:40:38.840870 15515 catalog_manager.cc:5495] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c reported cstate change: term changed from 0 to 1, leader changed from <none> to c9ec552b32f84d568fa60fea398a060c (127.8.151.193). New cstate: current_term: 1 leader_uuid: "c9ec552b32f84d568fa60fea398a060c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ec552b32f84d568fa60fea398a060c" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 35507 } health_report { overall_health: HEALTHY } } }
I20260928 06:40:38.852852 16252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.850057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42940 (local address 127.8.151.193:35507)
0928 06:40:38.850170 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:38.850173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:38.850186 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:38.850226 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:38.850228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:38.850268 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:38.850326 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:38.850579 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.851103 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.851971 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:38.852193 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:38.852260 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:38.852465 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:38.852530 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:38.852575 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:38.852673 (+ 98us) server_negotiation.cc:299] Negotiation successful
0928 06:40:38.852719 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 06:40:38.854775 16112 raft_consensus.cc:1261] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c [term 1 FOLLOWER]: Refusing update from remote peer c9ec552b32f84d568fa60fea398a060c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:38.854777 15975 raft_consensus.cc:1261] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f [term 1 FOLLOWER]: Refusing update from remote peer c9ec552b32f84d568fa60fea398a060c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:38.855252 16259 consensus_queue.cc:1035] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4c77749c17942f3bece23898e90320c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:38.855468 16266 consensus_queue.cc:1035] T bbdd43370f4947018549f060a49a5a85 P c9ec552b32f84d568fa60fea398a060c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2810e72ad0674063943bde466e9dc35f" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:38.856812 16265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.856506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.195:59753)
0928 06:40:38.856668 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.856732 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260928 06:40:38.858901 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.858570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.194:53507)
0928 06:40:38.858773 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.858829 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260928 06:40:38.861647 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.860974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.194:41287)
0928 06:40:38.861527 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.861580 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485}
I20260928 06:40:38.862303 16265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.861848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.195:34267)
0928 06:40:38.862180 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.862232 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270}
I20260928 06:40:38.864163 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.860272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.193:43775)
0928 06:40:38.864043 (+ 3771us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.864097 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3711}
I20260928 06:40:38.864812 15515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:38.864939 15515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:38.865017 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.864806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:33147 (local address 127.8.151.193:38971)
0928 06:40:38.864912 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.864958 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:33147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 06:40:38.868497 15515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f177e7daf9234317ba18e3de53eef1b7: Sending DeleteTablet for 3 replicas of tablet bbdd43370f4947018549f060a49a5a85
I20260928 06:40:38.869122 15955 tablet_service.cc:1514] Processing DeleteTablet for tablet bbdd43370f4947018549f060a49a5a85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:38 UTC) from {username='oryx', principal='oryx/127.8.151.253@KRBTEST.COM'} at 127.0.0.1:51812
I20260928 06:40:38.869207 15818 tablet_service.cc:1514] Processing DeleteTablet for tablet bbdd43370f4947018549f060a49a5a85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:38 UTC) from {username='oryx', principal='oryx/127.8.151.253@KRBTEST.COM'} at 127.0.0.1:42936
I20260928 06:40:38.869408 16092 tablet_service.cc:1514] Processing DeleteTablet for tablet bbdd43370f4947018549f060a49a5a85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:38 UTC) from {username='oryx', principal='oryx/127.8.151.253@KRBTEST.COM'} at 127.0.0.1:60310
I20260928 06:40:38.869716 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 15659
I20260928 06:40:38.869731 16278 tablet_replica.cc:330] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f: stopping tablet replica
I20260928 06:40:38.869853 16278 raft_consensus.cc:2229] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:38.870187 16278 raft_consensus.cc:2258] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:38.870910 16278 ts_tablet_manager.cc:1905] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:38.871011 16280 tablet_replica.cc:330] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c: stopping tablet replica
I20260928 06:40:38.871125 16280 raft_consensus.cc:2229] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:38.871272 16280 raft_consensus.cc:2258] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:38.871827 16280 ts_tablet_manager.cc:1905] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:38.873317 16278 ts_tablet_manager.cc:1918] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:38.873404 16278 log.cc:1196] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/bbdd43370f4947018549f060a49a5a85
I20260928 06:40:38.873638 16278 ts_tablet_manager.cc:1939] T bbdd43370f4947018549f060a49a5a85 P 2810e72ad0674063943bde466e9dc35f: Deleting consensus metadata
I20260928 06:40:38.873749 16280 ts_tablet_manager.cc:1918] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:38.873821 16280 log.cc:1196] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/bbdd43370f4947018549f060a49a5a85
I20260928 06:40:38.874033 16280 ts_tablet_manager.cc:1939] T bbdd43370f4947018549f060a49a5a85 P b4c77749c17942f3bece23898e90320c: Deleting consensus metadata
I20260928 06:40:38.874253 15501 catalog_manager.cc:4841] TS 2810e72ad0674063943bde466e9dc35f (127.8.151.194:39851): tablet bbdd43370f4947018549f060a49a5a85 (table test-table [id=a938d14aaf054ad7bbcd1cf98cd3287c]) successfully deleted
I20260928 06:40:38.874734 15502 catalog_manager.cc:4841] TS b4c77749c17942f3bece23898e90320c (127.8.151.195:46477): tablet bbdd43370f4947018549f060a49a5a85 (table test-table [id=a938d14aaf054ad7bbcd1cf98cd3287c]) successfully deleted
I20260928 06:40:38.877779 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 15892
I20260928 06:40:38.877976 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:38.877736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:35507 (local address 127.0.0.1:42948)
0928 06:40:38.877855 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:38.877904 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:35507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 06:40:38.878377 15499 catalog_manager.cc:4584] TS c9ec552b32f84d568fa60fea398a060c (127.8.151.193:35507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bbdd43370f4947018549f060a49a5a85: Network error: Client connection negotiation failed: client connection to 127.8.151.193:35507: connect: Connection refused (error 111)
I20260928 06:40:38.883685 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 16028
I20260928 06:40:38.888466 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 15379
I20260928 06:40:38.893340 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 15496
2026-09-28T06:40:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5046 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:38 dist-test-slave-gtr8 krb5kdc[16283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:38 dist-test-slave-gtr8 krb5kdc[16283](info): set up 2 sockets
Sep 28 06:40:38 dist-test-slave-gtr8 krb5kdc[16283](info): commencing operation
krb5kdc: starting...
W20260928 06:40:39.851908 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:39 dist-test-slave-gtr8 krb5kdc[16283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:39Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.254.keytab.
Entry for principal oryx/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.254.keytab.
I20260928 06:40:39.966873 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:38021
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.254.keytab
--principal=oryx/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:38021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:40.072330 16296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:40.072571 16296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:40.072608 16296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:40.076501 16296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:40.076560 16296 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 06:40:40.076575 16296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:40.076592 16296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:40.076606 16296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:40.076620 16296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:40.081736 16296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:38021
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.254.keytab
--principal=oryx/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:38021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:40.082707 16296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:40.083702 16296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:40.091723 16302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:40.091789 16296 server_base.cc:1034] running on GCE node
W20260928 06:40:40.091723 16301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:40.092003 16304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:40.092554 16296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:40.093600 16296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:40.094753 16296 hybrid_clock.cc:648] HybridClock initialized: now 1790577640094735 us; error 28 us; skew 500 ppm
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577640, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:40.097362 16296 init.cc:377] Logged in from keytab as oryx/127.8.151.254@KRBTEST.COM (short username oryx)
I20260928 06:40:40.098227 16296 webserver.cc:460] Webserver started at http://127.8.151.254:34299/ using document root <none> and password file <none>
I20260928 06:40:40.098752 16296 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:40.098815 16296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:40.099002 16296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:40.101150 16296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "2ca5783a377c4667a78b224630053cb5"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "1c580b3c6d5fabf6b9dd787388c91f59"
server_key_iv: "3de41704ab1b31850897f66767761c93"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.101680 16296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "2ca5783a377c4667a78b224630053cb5"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "1c580b3c6d5fabf6b9dd787388c91f59"
server_key_iv: "3de41704ab1b31850897f66767761c93"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.105307 16296 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:40.107707 16311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:40.108981 16296 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:40.109119 16296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "2ca5783a377c4667a78b224630053cb5"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "1c580b3c6d5fabf6b9dd787388c91f59"
server_key_iv: "3de41704ab1b31850897f66767761c93"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.109225 16296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:40.123526 16296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:40.126263 16296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:40.126433 16296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:40.133847 16296 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:38021
I20260928 06:40:40.133903 16373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:38021 every 8 connection(s)
I20260928 06:40:40.134778 16296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:40.137584 16374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:40.142661 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 16296
I20260928 06:40:40.142791 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:40.142993 16374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5: Bootstrap starting.
I20260928 06:40:40.143046 8799 external_mini_cluster.cc:1428] Setting key 36722116477581dc93f75259a2e33573
I20260928 06:40:40.145390 16374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:40.146227 16374 log.cc:824] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:40.148293 16374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5: No bootstrap required, opened a new log
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:40.152410 16374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca5783a377c4667a78b224630053cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38021 } }
I20260928 06:40:40.152649 16374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:40.152706 16374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca5783a377c4667a78b224630053cb5, State: Initialized, Role: FOLLOWER
I20260928 06:40:40.153188 16374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca5783a377c4667a78b224630053cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38021 } }
I20260928 06:40:40.153326 16374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:40.153383 16374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:40.153472 16374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:40.154278 16374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca5783a377c4667a78b224630053cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38021 } }
I20260928 06:40:40.154629 16374 leader_election.cc:302] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ca5783a377c4667a78b224630053cb5; no voters:
I20260928 06:40:40.154958 16374 leader_election.cc:288] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:40.155041 16379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:40.155258 16379 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [term 1 LEADER]: Becoming Leader. State: Replica: 2ca5783a377c4667a78b224630053cb5, State: Running, Role: LEADER
I20260928 06:40:40.155560 16379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca5783a377c4667a78b224630053cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38021 } }
I20260928 06:40:40.156263 16374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:40.156679 16379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ca5783a377c4667a78b224630053cb5. Latest consensus state: current_term: 1 leader_uuid: "2ca5783a377c4667a78b224630053cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca5783a377c4667a78b224630053cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38021 } } }
I20260928 06:40:40.156790 16379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:40.157351 16384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:40.157881 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.144274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58136 (local address 127.8.151.254:38021)
0928 06:40:40.144670 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.144686 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.144721 (+ 35us) server_negotiation.cc:407] Connection header received
0928 06:40:40.145338 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.145354 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.145701 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.145990 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:40.146836 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.147912 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.149075 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.149559 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.149673 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.152444 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:40.152450 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:40.152463 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:40.152490 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:40.154250 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.154715 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.154720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.154724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.154876 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.155247 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.155251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.155253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.155805 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:40.156005 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.156310 (+ 305us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.156774 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":111,"threads_started":1}
I20260928 06:40:40.157353 16381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ca5783a377c4667a78b224630053cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ca5783a377c4667a78b224630053cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38021 } } }
I20260928 06:40:40.158033 16381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5 [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:40.161332 16384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:40.166640 16384 catalog_manager.cc:1347] Generated new cluster ID: eaab8f75cefc4c03907ee70d5ce1ae95
I20260928 06:40:40.166693 16384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:40.180958 16384 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:40.182020 16384 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:40.186352 16384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5: Generated new TSK 0
I20260928 06:40:40.187038 16384 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab.
Entry for principal oryx/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab.
I20260928 06:40:40.218253 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38021
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab
--principal=oryx/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:40.324162 16400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:40.324397 16400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:40.324440 16400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:40.328274 16400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:40.328325 16400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:40.328361 16400 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 06:40:40.328390 16400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:40.333371 16400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab
--principal=oryx/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.151.254:38021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:40.334312 16400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:40.335278 16400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:40.344211 16405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:40.344204 16406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:40.344354 16400 server_base.cc:1034] running on GCE node
W20260928 06:40:40.344237 16408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:40.344902 16400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:40.345588 16400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:40.346740 16400 hybrid_clock.cc:648] HybridClock initialized: now 1790577640346713 us; error 40 us; skew 500 ppm
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577640, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:40.349287 16400 init.cc:377] Logged in from keytab as oryx/127.8.151.193@KRBTEST.COM (short username oryx)
I20260928 06:40:40.350095 16400 webserver.cc:460] Webserver started at http://127.8.151.193:42777/ using document root <none> and password file <none>
I20260928 06:40:40.350608 16400 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:40.350672 16400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:40.350844 16400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:40.352998 16400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "657d70a718194085addb9f170101a165"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "01b2fb699f7a44d431eb1e9f45520e23"
server_key_iv: "595e37ad5cb07191bc65577a64542e7b"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.353569 16400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "657d70a718194085addb9f170101a165"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "01b2fb699f7a44d431eb1e9f45520e23"
server_key_iv: "595e37ad5cb07191bc65577a64542e7b"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.357208 16400 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:40.359545 16415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:40.360683 16400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:40.360785 16400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "657d70a718194085addb9f170101a165"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "01b2fb699f7a44d431eb1e9f45520e23"
server_key_iv: "595e37ad5cb07191bc65577a64542e7b"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.360874 16400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:40.405405 16400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:40.408406 16400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:40.408592 16400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:40.410018 16400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:40.410068 16400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:40.410122 16400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:40.410152 16400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577640, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:40.420225 16400 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:38749
I20260928 06:40:40.420372 16530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:38749 every 8 connection(s)
I20260928 06:40:40.421099 16400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:40.424420 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 16400
I20260928 06:40:40.424497 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.411011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:36123 (local address 127.8.151.254:38021)
0928 06:40:40.411176 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.411181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.412069 (+ 888us) server_negotiation.cc:407] Connection header received
0928 06:40:40.413510 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.413514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.413577 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.413677 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:40.415218 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.415894 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.417535 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.417831 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.417924 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.420873 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:40.420877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:40.420879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:40.420906 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:40.422413 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.422908 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.422912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.422915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.422996 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.423376 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.423379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.423381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.423571 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:40.423666 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.424156 (+ 490us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.424312 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:40:40.424548 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:40:40.424815 8799 external_mini_cluster.cc:1428] Setting key 2b98d143b5506efe1bc134b56f782409
I20260928 06:40:40.425226 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.411423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38021 (local address 127.8.151.193:36123)
0928 06:40:40.411897 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.411955 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.413235 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.413866 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.413875 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.414336 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:40.415017 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.415038 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.416008 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.416012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.417406 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.417417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.417997 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.418001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.418210 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.418874 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:40.418894 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:40.420695 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:40.422536 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.422542 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.422555 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.422799 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.423080 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.423084 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.423086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.423268 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.423675 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:40.423681 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:40.423917 (+ 236us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.424076 (+ 159us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.424371 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":98,"threads_started":1}
I20260928 06:40:40.426393 16531 heartbeater.cc:347] Connected to a master server at 127.8.151.254:38021
I20260928 06:40:40.426525 16531 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:40.426975 16531 heartbeater.cc:511] Master 127.8.151.254:38021 requested a full tablet report, sending...
I20260928 06:40:40.428459 16326 ts_manager.cc:194] Registered new tserver with Master: 657d70a718194085addb9f170101a165 (127.8.151.193:38749)
I20260928 06:40:40.429670 16326 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:36123
WARNING: no policy specified for oryx/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:40:40.437211 16538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.430300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38021 (local address 127.8.151.193:51165)
0928 06:40:40.430540 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.430554 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.430664 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.430882 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.430885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.431095 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:40.431226 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.431233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.432165 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.432167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.432704 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.432711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.433230 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.433232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.433296 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.433733 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:40.433744 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:40.434039 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:40.435687 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.435689 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.435691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.435915 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.436191 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.436193 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.436195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.436289 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.436732 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:40.436736 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:40.436857 (+ 121us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.436957 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.437062 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":89,"threads_started":1}
I20260928 06:40:40.437248 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.430393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:51165 (local address 127.8.151.254:38021)
0928 06:40:40.430498 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.430502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.430620 (+ 118us) server_negotiation.cc:407] Connection header received
0928 06:40:40.430740 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.430743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.430782 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.430855 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:40.431326 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.432054 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.432810 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.433130 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.433234 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.434162 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:40.434166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:40.434168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:40.434192 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:40.435573 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.436010 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.436014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.436015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.436093 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.436428 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.436431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.436432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.436599 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:40.436704 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.437038 (+ 334us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.437133 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab.
Entry for principal oryx/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab.
I20260928 06:40:40.453935 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38021
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab
--principal=oryx/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:40.559077 16540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:40.559370 16540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:40.559422 16540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:40.563361 16540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:40.563467 16540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:40.563506 16540 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 06:40:40.563537 16540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:40.568653 16540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab
--principal=oryx/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.151.254:38021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:40.569657 16540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:40.570645 16540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:40.579787 16548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:40.579771 16545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:40.579787 16546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:40.579913 16540 server_base.cc:1034] running on GCE node
I20260928 06:40:40.580324 16540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:40.580907 16540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:40.582067 16540 hybrid_clock.cc:648] HybridClock initialized: now 1790577640582047 us; error 31 us; skew 500 ppm
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577640, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:40.588371 16540 init.cc:377] Logged in from keytab as oryx/127.8.151.194@KRBTEST.COM (short username oryx)
I20260928 06:40:40.589195 16540 webserver.cc:460] Webserver started at http://127.8.151.194:45739/ using document root <none> and password file <none>
I20260928 06:40:40.589702 16540 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:40.589763 16540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:40.589965 16540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:40.591928 16540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "19fefecd803f4948ab3dc98083a5d7fe"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "860a98cc5cec897d361fd37dc8a5213b"
server_key_iv: "63e02ae40091e010b2aaced31c455ce2"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.592437 16540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "19fefecd803f4948ab3dc98083a5d7fe"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "860a98cc5cec897d361fd37dc8a5213b"
server_key_iv: "63e02ae40091e010b2aaced31c455ce2"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.595880 16540 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 06:40:40.598145 16555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:40.599313 16540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:40.599416 16540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "19fefecd803f4948ab3dc98083a5d7fe"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "860a98cc5cec897d361fd37dc8a5213b"
server_key_iv: "63e02ae40091e010b2aaced31c455ce2"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.599520 16540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:40.610427 16540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:40.613132 16540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:40.613291 16540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:40.614646 16540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:40.614693 16540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:40.614742 16540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:40.614769 16540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577640, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:40.624723 16540 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:39967
I20260928 06:40:40.624766 16670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:39967 every 8 connection(s)
I20260928 06:40:40.625610 16540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:40.627492 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.615444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:36351 (local address 127.8.151.254:38021)
0928 06:40:40.615552 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.615557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.616570 (+ 1013us) server_negotiation.cc:407] Connection header received
0928 06:40:40.617571 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.617574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.617635 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.617693 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:40.619006 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.619834 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.621072 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.621299 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.621387 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.624128 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:40.624132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:40.624135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:40.624161 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:40.625509 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.626008 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.626012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.626014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.626099 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.626441 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.626445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.626447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.626677 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:40.626757 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.627249 (+ 492us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.627343 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 06:40:40.628268 16564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.615965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38021 (local address 127.8.151.194:36351)
0928 06:40:40.616428 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.616468 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.617320 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.618040 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.618046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.618343 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:40.618861 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.618880 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.619962 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.619966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.620929 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.620938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.621450 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.621455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.621629 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.622135 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:40.622154 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:40.623966 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:40.625614 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.625619 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.625632 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.625897 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.626185 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.626188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.626190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.626342 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.626769 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:40.626774 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:40.626980 (+ 206us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.627219 (+ 239us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.627450 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":119,"threads_started":1}
I20260928 06:40:40.629292 16671 heartbeater.cc:347] Connected to a master server at 127.8.151.254:38021
I20260928 06:40:40.629417 16671 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:40.629741 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 16540
I20260928 06:40:40.629858 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:40.629941 16671 heartbeater.cc:511] Master 127.8.151.254:38021 requested a full tablet report, sending...
I20260928 06:40:40.630173 8799 external_mini_cluster.cc:1428] Setting key ac20b2e676c6a3571c35f957e28f0b11
I20260928 06:40:40.630971 16328 ts_manager.cc:194] Registered new tserver with Master: 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967)
I20260928 06:40:40.631686 16328 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:36351
I20260928 06:40:40.639057 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.632297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38021 (local address 127.8.151.194:60501)
0928 06:40:40.632577 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.632590 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.632681 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.632904 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.632907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.633098 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:40.633208 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.633215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.634166 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.634168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.634706 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.634714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.635268 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.635271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.635341 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.635805 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:40.635815 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:40.636100 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:40.637744 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.637747 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.637748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.637919 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.638179 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.638180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.638182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.638247 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.638673 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:40.638675 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:40.638744 (+ 69us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.638816 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.638914 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":86,"threads_started":1}
I20260928 06:40:40.639205 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.632166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:60501 (local address 127.8.151.254:38021)
0928 06:40:40.632274 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.632277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.632664 (+ 387us) server_negotiation.cc:407] Connection header received
0928 06:40:40.632764 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.632767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.632809 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.632876 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:40.633311 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.634036 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.634809 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.635128 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.635264 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.636232 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:40.636236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:40.636239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:40.636265 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:40.637648 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.638007 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.638010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.638012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.638101 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.638318 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.638321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.638322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.638588 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:40.638689 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.638893 (+ 204us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.639016 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for oryx/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab.
Entry for principal oryx/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab.
I20260928 06:40:40.657612 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38021
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab
--principal=oryx/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:40.766423 16680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:40.766711 16680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:40.766760 16680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:40.770821 16680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:40.770902 16680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:40.770944 16680 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 06:40:40.770980 16680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:40.776615 16680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab
--principal=oryx/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.151.254:38021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:40.777616 16680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:40.778584 16680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:40.787962 16685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:40.788017 16680 server_base.cc:1034] running on GCE node
W20260928 06:40:40.787969 16686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:40.787981 16688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:40.788555 16680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:40.789192 16680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:40.790369 16680 hybrid_clock.cc:648] HybridClock initialized: now 1790577640790351 us; error 30 us; skew 500 ppm
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577640, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:40.793129 16680 init.cc:377] Logged in from keytab as oryx/127.8.151.195@KRBTEST.COM (short username oryx)
I20260928 06:40:40.794024 16680 webserver.cc:460] Webserver started at http://127.8.151.195:41233/ using document root <none> and password file <none>
I20260928 06:40:40.794543 16680 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:40.794605 16680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:40.794807 16680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:40.796852 16680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "a48ef9cac41346eb997d8dc80006d7de"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "18c1e236870a9c5edce3ddf91364506b"
server_key_iv: "97c0895145e5ad9086a51a597d670af1"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.797381 16680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "a48ef9cac41346eb997d8dc80006d7de"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "18c1e236870a9c5edce3ddf91364506b"
server_key_iv: "97c0895145e5ad9086a51a597d670af1"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.800957 16680 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:40:40.803490 16695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:40.804728 16680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:40:40.804842 16680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "a48ef9cac41346eb997d8dc80006d7de"
format_stamp: "Formatted at 2026-09-28 06:40:40 on dist-test-slave-gtr8"
server_key: "18c1e236870a9c5edce3ddf91364506b"
server_key_iv: "97c0895145e5ad9086a51a597d670af1"
server_key_version: "encryptionkey@0"
I20260928 06:40:40.804944 16680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:40.815462 16680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:40.818169 16680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:40.818341 16680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:40.819804 16680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:40.819854 16680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:40.819905 16680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:40.819932 16680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577640, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:40.830022 16680 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:46701
I20260928 06:40:40.830654 16811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:46701 every 8 connection(s)
I20260928 06:40:40.830868 16680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:40.832473 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.820661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:35629 (local address 127.8.151.254:38021)
0928 06:40:40.820780 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.820783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.821735 (+ 952us) server_negotiation.cc:407] Connection header received
0928 06:40:40.822724 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.822728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.822787 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.822844 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:40.824390 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.824964 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.826059 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.826350 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.826469 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.829057 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:40.829061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:40.829064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:40.829091 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:40.830514 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.831062 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.831065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.831067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.831172 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.831506 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.831509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.831511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.831728 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:40.831824 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.832257 (+ 433us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.832355 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:40:40.833178 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 16680
I20260928 06:40:40.833238 16704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.821108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38021 (local address 127.8.151.195:35629)
0928 06:40:40.821593 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.821635 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.822476 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.823248 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.823258 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.823712 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:40.824254 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.824270 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.825077 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.825081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.825933 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.825941 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.826533 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.826538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.826748 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.827417 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:40.827436 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:40.828903 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:40.830619 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.830625 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.830640 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.830966 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.831243 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.831246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.831248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.831402 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.831827 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:40.831832 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:40.832040 (+ 208us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.832194 (+ 154us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.832429 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":122,"threads_started":1}
I20260928 06:40:40.833302 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:40.833622 8799 external_mini_cluster.cc:1428] Setting key 32ebc81cad20b674f6c9f7d3394e7a41
I20260928 06:40:40.834607 16812 heartbeater.cc:347] Connected to a master server at 127.8.151.254:38021
I20260928 06:40:40.834746 16812 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:40.835332 16812 heartbeater.cc:511] Master 127.8.151.254:38021 requested a full tablet report, sending...
I20260928 06:40:40.836326 16328 ts_manager.cc:194] Registered new tserver with Master: a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701)
I20260928 06:40:40.837045 16328 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:35629
I20260928 06:40:40.843165 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.836673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38021 (local address 127.8.151.195:56377)
0928 06:40:40.836943 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.836956 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.837059 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.837291 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.837293 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.837500 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:40.837619 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.837626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.838604 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.838607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.839253 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.839262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.839643 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.839646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.839712 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.840169 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:40.840180 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:40.840467 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:40.841914 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.841916 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.841918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.842117 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.842356 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:40.842359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:40.842360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:40.842444 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:40.842739 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:40.842741 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:40.842820 (+ 79us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.842896 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.843001 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":96,"threads_started":1}
I20260928 06:40:40.843173 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.836767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:56377 (local address 127.8.151.254:38021)
0928 06:40:40.836879 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.836884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.837018 (+ 134us) server_negotiation.cc:407] Connection header received
0928 06:40:40.837144 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.837147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.837191 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.837260 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:40.837722 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.838467 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.839349 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.839553 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.839635 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.840590 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:40.840592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:40.840594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:40.840614 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:40.841818 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.842188 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.842190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.842192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.842255 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.842519 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.842521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.842522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.842669 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:40.842739 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.842931 (+ 192us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.843013 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:40:40.847286 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577640, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 06:40:40.869035 16377 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:40.863484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58140 (local address 127.8.151.254:38021)
0928 06:40:40.863597 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.863601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.863614 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:40.863654 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.863656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.863696 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.863764 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:40.864435 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.864957 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.865800 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.866004 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.866084 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.868881 (+ 2797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58140: BlockingRecv error: recv got EOF from 127.0.0.1:58140 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 28 06:40:40 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577640, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:40.877115 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.870265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58148 (local address 127.8.151.254:38021)
0928 06:40:40.870362 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.870365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.870451 (+ 86us) server_negotiation.cc:407] Connection header received
0928 06:40:40.870531 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.870533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.870567 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.870649 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:40.871151 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.871660 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.872391 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.872600 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.872668 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.874596 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:40.874599 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:40.874601 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:40.874623 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:40.875878 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.876232 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.876234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.876236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.876301 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:40.876479 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:40.876481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:40.876482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:40.876653 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:40.876744 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.876914 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.876996 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 06:40:40.879768 16328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:40.882596 16328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:40.895963 16838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.890968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:39967 (local address 127.0.0.1:49722)
0928 06:40:40.891508 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.891521 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.891593 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.892461 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.892464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.892482 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:40.892554 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.892559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.893663 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.893666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.894489 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.894497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.895427 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.895430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.895541 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.895590 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.895693 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.895815 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260928 06:40:40.895985 16839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.891375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49722 (local address 127.8.151.194:39967)
0928 06:40:40.892062 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.892073 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.892093 (+ 20us) server_negotiation.cc:407] Connection header received
0928 06:40:40.892161 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.892165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.892323 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.892462 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:40.892642 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.893551 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.894607 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.895327 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.895420 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.895575 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.895737 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.895829 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":85,"threads_started":1}
I20260928 06:40:40.896275 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.890914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48110 (local address 127.8.151.195:46701)
0928 06:40:40.891322 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.891332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.891352 (+ 20us) server_negotiation.cc:407] Connection header received
0928 06:40:40.891420 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.891425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.891654 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.891785 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:40.892047 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.893224 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.894564 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.895358 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.895456 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.895601 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.896033 (+ 432us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.896144 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":107,"threads_started":1}
I20260928 06:40:40.896476 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.890449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34182 (local address 127.8.151.193:38749)
0928 06:40:40.890731 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.890748 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.891101 (+ 353us) server_negotiation.cc:407] Connection header received
0928 06:40:40.891355 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.891360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.891580 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.891697 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:40.892093 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.892946 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.894502 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.895358 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.895458 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.895601 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.895932 (+ 331us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.896020 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":88,"threads_started":1}
I20260928 06:40:40.896517 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.890439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:46701 (local address 127.0.0.1:48110)
0928 06:40:40.890971 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.891022 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.891185 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.891777 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.891783 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.891804 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:40.891925 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.891932 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.893333 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.893336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.894451 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.894459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.895646 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.895650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.895810 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.895831 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.895897 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.895953 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":49,"thread_start_us":99,"threads_started":1}
I20260928 06:40:40.896859 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.890302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38749 (local address 127.0.0.1:34182)
0928 06:40:40.890971 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.891019 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.891195 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.891856 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.891859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.891873 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:40.891980 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.891987 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.893044 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.893046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.894359 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.894369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.895662 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.895665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.895783 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.895806 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.896651 (+ 845us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.896727 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":26,"thread_start_us":78,"threads_started":1}
I20260928 06:40:40.898880 16605 tablet_service.cc:1467] Processing CreateTablet for tablet cb9a04c0d7c04a77866e4e029b9c2cd2 (DEFAULT_TABLE table=test-table [id=a600801f370f4933b1a6b71fec25a912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:40.899190 16745 tablet_service.cc:1467] Processing CreateTablet for tablet cb9a04c0d7c04a77866e4e029b9c2cd2 (DEFAULT_TABLE table=test-table [id=a600801f370f4933b1a6b71fec25a912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:40.899185 16453 tablet_service.cc:1467] Processing CreateTablet for tablet cb9a04c0d7c04a77866e4e029b9c2cd2 (DEFAULT_TABLE table=test-table [id=a600801f370f4933b1a6b71fec25a912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:40.899881 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9a04c0d7c04a77866e4e029b9c2cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:40.900161 16745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9a04c0d7c04a77866e4e029b9c2cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:40.900161 16453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9a04c0d7c04a77866e4e029b9c2cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:40.906754 16840 tablet_bootstrap.cc:492] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: Bootstrap starting.
I20260928 06:40:40.907807 16841 tablet_bootstrap.cc:492] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: Bootstrap starting.
I20260928 06:40:40.908927 16840 tablet_bootstrap.cc:654] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:40.909482 16841 tablet_bootstrap.cc:654] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:40.909626 16840 log.cc:824] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:40.910199 16842 tablet_bootstrap.cc:492] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: Bootstrap starting.
I20260928 06:40:40.910215 16841 log.cc:824] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:40.911566 16840 tablet_bootstrap.cc:492] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: No bootstrap required, opened a new log
I20260928 06:40:40.911755 16840 ts_tablet_manager.cc:1397] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260928 06:40:40.912099 16842 tablet_bootstrap.cc:654] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:40.912232 16841 tablet_bootstrap.cc:492] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: No bootstrap required, opened a new log
I20260928 06:40:40.912412 16841 ts_tablet_manager.cc:1397] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260928 06:40:40.912770 16842 log.cc:824] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:40.914255 16842 tablet_bootstrap.cc:492] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: No bootstrap required, opened a new log
I20260928 06:40:40.914417 16842 ts_tablet_manager.cc:1397] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260928 06:40:40.915725 16840 raft_consensus.cc:348] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } }
I20260928 06:40:40.916005 16840 raft_consensus.cc:374] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:40.916059 16840 raft_consensus.cc:729] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19fefecd803f4948ab3dc98083a5d7fe, State: Initialized, Role: FOLLOWER
I20260928 06:40:40.916213 16841 raft_consensus.cc:348] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } }
I20260928 06:40:40.916451 16841 raft_consensus.cc:374] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:40.916501 16841 raft_consensus.cc:729] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657d70a718194085addb9f170101a165, State: Initialized, Role: FOLLOWER
I20260928 06:40:40.916492 16840 consensus_queue.cc:260] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } }
I20260928 06:40:40.916949 16841 consensus_queue.cc:260] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } }
I20260928 06:40:40.917346 16671 heartbeater.cc:503] Master 127.8.151.254:38021 was elected leader, sending a full tablet report...
I20260928 06:40:40.917507 16840 ts_tablet_manager.cc:1428] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260928 06:40:40.917409 16842 raft_consensus.cc:348] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } }
I20260928 06:40:40.917632 16842 raft_consensus.cc:374] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:40.917675 16842 raft_consensus.cc:729] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48ef9cac41346eb997d8dc80006d7de, State: Initialized, Role: FOLLOWER
I20260928 06:40:40.917794 16841 ts_tablet_manager.cc:1428] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:40:40.918089 16531 heartbeater.cc:503] Master 127.8.151.254:38021 was elected leader, sending a full tablet report...
I20260928 06:40:40.918046 16842 consensus_queue.cc:260] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } }
I20260928 06:40:40.919217 16842 ts_tablet_manager.cc:1428] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260928 06:40:40.919515 16812 heartbeater.cc:503] Master 127.8.151.254:38021 was elected leader, sending a full tablet report...
W20260928 06:40:40.922408 16532 tablet.cc:2369] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:40.942095 16848 raft_consensus.cc:482] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:40.942291 16848 raft_consensus.cc:504] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } }
I20260928 06:40:40.943393 16848 leader_election.cc:288] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701), 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967)
I20260928 06:40:40.946990 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.943695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:44649 (local address 127.8.151.195:46701)
0928 06:40:40.943815 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.943820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.943990 (+ 170us) server_negotiation.cc:407] Connection header received
0928 06:40:40.944145 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.944147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.944188 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.944252 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:40.944537 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.945044 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.946130 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.946506 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.946615 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.946665 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.946811 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.946872 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 06:40:40.947019 16839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.943717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:55525 (local address 127.8.151.194:39967)
0928 06:40:40.943841 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.943846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.943978 (+ 132us) server_negotiation.cc:407] Connection header received
0928 06:40:40.944146 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.944148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.944186 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.944244 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:40.944523 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.945033 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.946136 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.946510 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.946618 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.946666 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.946832 (+ 166us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.946899 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:40:40.947052 16850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.943588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:39967 (local address 127.8.151.193:55525)
0928 06:40:40.943903 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.943914 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.944046 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.944266 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.944269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.944294 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:40.944430 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.944435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.945137 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.945140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.946013 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.946022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.946628 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.946632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.946733 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.946751 (+ 18us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.946810 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.946878 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":69,"spinlock_wait_cycles":66816,"thread_start_us":88,"threads_started":1}
I20260928 06:40:40.947062 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.943588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:46701 (local address 127.8.151.193:44649)
0928 06:40:40.943910 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:40.943920 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:40.944048 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:40.944269 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:40.944272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:40.944297 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:40.944438 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.944443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.945138 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.945142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.946022 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:40.946031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.946615 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.946619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:40.946711 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.946731 (+ 20us) client_negotiation.cc:769] Sending connection context
0928 06:40:40.946791 (+ 60us) client_negotiation.cc:240] Negotiation successful
0928 06:40:40.946860 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":127,"spinlock_wait_cycles":12800,"thread_start_us":59,"threads_started":1}
I20260928 06:40:40.947667 16625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9a04c0d7c04a77866e4e029b9c2cd2" candidate_uuid: "657d70a718194085addb9f170101a165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19fefecd803f4948ab3dc98083a5d7fe" is_pre_election: true
I20260928 06:40:40.947732 16747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9a04c0d7c04a77866e4e029b9c2cd2" candidate_uuid: "657d70a718194085addb9f170101a165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48ef9cac41346eb997d8dc80006d7de" is_pre_election: true
I20260928 06:40:40.947938 16625 raft_consensus.cc:2454] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657d70a718194085addb9f170101a165 in term 0.
I20260928 06:40:40.947983 16747 raft_consensus.cc:2454] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657d70a718194085addb9f170101a165 in term 0.
I20260928 06:40:40.948378 16417 leader_election.cc:302] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fefecd803f4948ab3dc98083a5d7fe, 657d70a718194085addb9f170101a165; no voters:
I20260928 06:40:40.948622 16848 raft_consensus.cc:2781] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:40.948697 16848 raft_consensus.cc:482] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:40.948728 16848 raft_consensus.cc:3037] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:40.949692 16848 raft_consensus.cc:504] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } }
I20260928 06:40:40.949999 16848 leader_election.cc:288] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [CANDIDATE]: Term 1 election: Requested vote from peers a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701), 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967)
I20260928 06:40:40.950279 16747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9a04c0d7c04a77866e4e029b9c2cd2" candidate_uuid: "657d70a718194085addb9f170101a165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48ef9cac41346eb997d8dc80006d7de"
I20260928 06:40:40.950281 16625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9a04c0d7c04a77866e4e029b9c2cd2" candidate_uuid: "657d70a718194085addb9f170101a165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19fefecd803f4948ab3dc98083a5d7fe"
I20260928 06:40:40.950394 16747 raft_consensus.cc:3037] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:40.950395 16625 raft_consensus.cc:3037] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:40.951426 16747 raft_consensus.cc:2454] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657d70a718194085addb9f170101a165 in term 1.
I20260928 06:40:40.951735 16419 leader_election.cc:302] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 657d70a718194085addb9f170101a165, a48ef9cac41346eb997d8dc80006d7de; no voters:
I20260928 06:40:40.951797 16625 raft_consensus.cc:2454] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657d70a718194085addb9f170101a165 in term 1.
I20260928 06:40:40.951905 16848 raft_consensus.cc:2781] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:40.952086 16848 raft_consensus.cc:686] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 1 LEADER]: Becoming Leader. State: Replica: 657d70a718194085addb9f170101a165, State: Running, Role: LEADER
I20260928 06:40:40.952414 16848 consensus_queue.cc:237] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } }
I20260928 06:40:40.955232 16327 catalog_manager.cc:5495] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657d70a718194085addb9f170101a165 (127.8.151.193). New cstate: current_term: 1 leader_uuid: "657d70a718194085addb9f170101a165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } health_report { overall_health: UNKNOWN } } }
I20260928 06:40:40.973801 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:40.970415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34194 (local address 127.8.151.193:38749)
0928 06:40:40.970543 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:40.970546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:40.970560 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:40:40.970665 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:40.970667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:40.970722 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:40.970802 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:40.970973 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.971511 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.972327 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:40.972556 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:40.972644 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:40.972956 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:40.973382 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:40.973526 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:40.973603 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 06:40:40.973655 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 06:40:40.979449 16625 raft_consensus.cc:1261] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Refusing update from remote peer 657d70a718194085addb9f170101a165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:40.979449 16747 raft_consensus.cc:1261] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Refusing update from remote peer 657d70a718194085addb9f170101a165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:40.980082 16848 consensus_queue.cc:1035] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:40.980260 16852 consensus_queue.cc:1035] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:40.987982 16855 mvcc.cc:205] Tried to move back new op lower bound from 7334206017445281792 to 7334206017343287296. Current Snapshot: MvccSnapshot[applied={T|T < 7334206017445281792}]
I20260928 06:40:40.989471 16856 mvcc.cc:205] Tried to move back new op lower bound from 7334206017445281792 to 7334206017343287296. Current Snapshot: MvccSnapshot[applied={T|T < 7334206017445281792}]
I20260928 06:40:40.998042 16327 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:40.998196 16327 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:41.000368 16327 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5: Sending DeleteTablet for 3 replicas of tablet cb9a04c0d7c04a77866e4e029b9c2cd2
I20260928 06:40:41.001076 16453 tablet_service.cc:1514] Processing DeleteTablet for tablet cb9a04c0d7c04a77866e4e029b9c2cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:40 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:34182
I20260928 06:40:41.001259 16605 tablet_service.cc:1514] Processing DeleteTablet for tablet cb9a04c0d7c04a77866e4e029b9c2cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:40 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:49722
I20260928 06:40:41.001276 16745 tablet_service.cc:1514] Processing DeleteTablet for tablet cb9a04c0d7c04a77866e4e029b9c2cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:40 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48110
I20260928 06:40:41.001518 16864 tablet_replica.cc:330] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: stopping tablet replica
I20260928 06:40:41.001801 16864 raft_consensus.cc:2229] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:41.002041 16866 tablet_replica.cc:330] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: stopping tablet replica
I20260928 06:40:41.001837 16865 tablet_replica.cc:330] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: stopping tablet replica
I20260928 06:40:41.002233 16864 raft_consensus.cc:2258] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:41.002290 16327 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:41.002713 16327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:41.002918 16865 raft_consensus.cc:2229] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:41.002761 16866 raft_consensus.cc:2229] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:41.003368 16865 raft_consensus.cc:2258] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:41.003691 16866 raft_consensus.cc:2258] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:41.003691 16864 ts_tablet_manager.cc:1905] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:41.005352 16866 ts_tablet_manager.cc:1905] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:41.005398 16865 ts_tablet_manager.cc:1905] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:41.007625 16453 tablet_service.cc:1467] Processing CreateTablet for tablet 4a476d2a504a47899d0cd1e2604d64fb (DEFAULT_TABLE table=test-table [id=9bddb8387c88417da1bb6da088178abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:41.007893 16453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a476d2a504a47899d0cd1e2604d64fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.008287 16866 ts_tablet_manager.cc:1918] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:41.008451 16866 log.cc:1196] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/cb9a04c0d7c04a77866e4e029b9c2cd2
I20260928 06:40:41.008803 16866 ts_tablet_manager.cc:1939] T cb9a04c0d7c04a77866e4e029b9c2cd2 P a48ef9cac41346eb997d8dc80006d7de: Deleting consensus metadata
I20260928 06:40:41.009629 16865 ts_tablet_manager.cc:1918] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:41.009724 16865 log.cc:1196] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/cb9a04c0d7c04a77866e4e029b9c2cd2
I20260928 06:40:41.009992 16865 ts_tablet_manager.cc:1939] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 19fefecd803f4948ab3dc98083a5d7fe: Deleting consensus metadata
I20260928 06:40:41.010171 16841 tablet_bootstrap.cc:492] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165: Bootstrap starting.
I20260928 06:40:41.011113 16841 tablet_bootstrap.cc:654] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.011512 16313 catalog_manager.cc:4841] TS 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967): tablet cb9a04c0d7c04a77866e4e029b9c2cd2 (table test-table [id=a600801f370f4933b1a6b71fec25a912]) successfully deleted
I20260928 06:40:41.011679 16745 tablet_service.cc:1467] Processing CreateTablet for tablet 4a476d2a504a47899d0cd1e2604d64fb (DEFAULT_TABLE table=test-table [id=9bddb8387c88417da1bb6da088178abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:41.011905 16745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a476d2a504a47899d0cd1e2604d64fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.011909 16315 catalog_manager.cc:4841] TS a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701): tablet cb9a04c0d7c04a77866e4e029b9c2cd2 (table test-table [id=a600801f370f4933b1a6b71fec25a912]) successfully deleted
I20260928 06:40:41.011972 16605 tablet_service.cc:1467] Processing CreateTablet for tablet 4a476d2a504a47899d0cd1e2604d64fb (DEFAULT_TABLE table=test-table [id=9bddb8387c88417da1bb6da088178abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:41.012214 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a476d2a504a47899d0cd1e2604d64fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.012863 16841 tablet_bootstrap.cc:492] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165: No bootstrap required, opened a new log
I20260928 06:40:41.012940 16841 ts_tablet_manager.cc:1397] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 06:40:41.013396 16841 raft_consensus.cc:348] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.013495 16841 raft_consensus.cc:374] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.013525 16841 raft_consensus.cc:729] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657d70a718194085addb9f170101a165, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.013644 16841 consensus_queue.cc:260] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.013898 16841 ts_tablet_manager.cc:1428] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:41.013901 16842 tablet_bootstrap.cc:492] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: Bootstrap starting.
I20260928 06:40:41.014364 16840 tablet_bootstrap.cc:492] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: Bootstrap starting.
I20260928 06:40:41.014783 16842 tablet_bootstrap.cc:654] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.015267 16840 tablet_bootstrap.cc:654] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.015934 16864 ts_tablet_manager.cc:1918] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:41.016019 16842 tablet_bootstrap.cc:492] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: No bootstrap required, opened a new log
I20260928 06:40:41.016036 16864 log.cc:1196] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/cb9a04c0d7c04a77866e4e029b9c2cd2
I20260928 06:40:41.016093 16842 ts_tablet_manager.cc:1397] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:41.016314 16864 ts_tablet_manager.cc:1939] T cb9a04c0d7c04a77866e4e029b9c2cd2 P 657d70a718194085addb9f170101a165: Deleting consensus metadata
I20260928 06:40:41.016515 16842 raft_consensus.cc:348] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.016624 16842 raft_consensus.cc:374] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.016656 16842 raft_consensus.cc:729] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48ef9cac41346eb997d8dc80006d7de, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.016767 16842 consensus_queue.cc:260] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.016856 16840 tablet_bootstrap.cc:492] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: No bootstrap required, opened a new log
I20260928 06:40:41.016918 16840 ts_tablet_manager.cc:1397] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260928 06:40:41.017005 16842 ts_tablet_manager.cc:1428] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:41.017328 16840 raft_consensus.cc:348] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.017427 16840 raft_consensus.cc:374] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.017457 16840 raft_consensus.cc:729] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19fefecd803f4948ab3dc98083a5d7fe, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.017557 16840 consensus_queue.cc:260] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.017792 16840 ts_tablet_manager.cc:1428] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260928 06:40:41.018141 16313 catalog_manager.cc:4841] TS 657d70a718194085addb9f170101a165 (127.8.151.193:38749): tablet cb9a04c0d7c04a77866e4e029b9c2cd2 (table test-table [id=a600801f370f4933b1a6b71fec25a912]) successfully deleted
W20260928 06:40:41.081940 16813 tablet.cc:2369] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 06:40:41.126895 16672 tablet.cc:2369] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:40:41.209153 16847 raft_consensus.cc:482] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:41.209328 16847 raft_consensus.cc:504] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.210476 16847 leader_election.cc:288] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657d70a718194085addb9f170101a165 (127.8.151.193:38749), a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701)
I20260928 06:40:41.214342 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.210879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:55451 (local address 127.8.151.195:46701)
0928 06:40:41.211079 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.211083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.211157 (+ 74us) server_negotiation.cc:407] Connection header received
0928 06:40:41.211310 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.211313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.211357 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.211410 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:41.211695 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.212218 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.213362 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.213764 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.213862 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.213933 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.214147 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.214215 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260928 06:40:41.214353 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.210820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:34047 (local address 127.8.151.193:38749)
0928 06:40:41.210962 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.210966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.211205 (+ 239us) server_negotiation.cc:407] Connection header received
0928 06:40:41.211324 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.211326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.211368 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.211432 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:41.211685 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.212218 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.213371 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.213783 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.213885 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.213937 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.214167 (+ 230us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.214242 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 06:40:41.214366 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.210729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:46701 (local address 127.8.151.194:55451)
0928 06:40:41.211058 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:41.211072 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:41.211198 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:41.211436 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:41.211439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:41.211454 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:41.211593 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.211599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.212316 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.212319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.213258 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.213267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.213871 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.213874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.213966 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.214037 (+ 71us) client_negotiation.cc:769] Sending connection context
0928 06:40:41.214128 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 06:40:41.214207 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":77,"spinlock_wait_cycles":22912,"thread_start_us":85,"threads_started":1}
I20260928 06:40:41.214372 16871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.210729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38749 (local address 127.8.151.194:34047)
0928 06:40:41.211105 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:41.211119 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:41.211238 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:41.211456 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:41.211459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:41.211499 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:41.211589 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.211594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.212311 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.212314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.213257 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.213268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.213885 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.213888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.214004 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.214040 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 06:40:41.214141 (+ 101us) client_negotiation.cc:240] Negotiation successful
0928 06:40:41.214216 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":75,"spinlock_wait_cycles":2176,"thread_start_us":68,"threads_started":1}
I20260928 06:40:41.214757 16747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a476d2a504a47899d0cd1e2604d64fb" candidate_uuid: "19fefecd803f4948ab3dc98083a5d7fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48ef9cac41346eb997d8dc80006d7de" is_pre_election: true
I20260928 06:40:41.214890 16747 raft_consensus.cc:2454] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19fefecd803f4948ab3dc98083a5d7fe in term 0.
I20260928 06:40:41.215270 16559 leader_election.cc:302] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fefecd803f4948ab3dc98083a5d7fe, a48ef9cac41346eb997d8dc80006d7de; no voters:
I20260928 06:40:41.215425 16485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a476d2a504a47899d0cd1e2604d64fb" candidate_uuid: "19fefecd803f4948ab3dc98083a5d7fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657d70a718194085addb9f170101a165" is_pre_election: true
I20260928 06:40:41.215512 16847 raft_consensus.cc:2781] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:41.215560 16847 raft_consensus.cc:482] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:41.215602 16847 raft_consensus.cc:3037] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.215659 16485 raft_consensus.cc:2454] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19fefecd803f4948ab3dc98083a5d7fe in term 0.
I20260928 06:40:41.216621 16847 raft_consensus.cc:504] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.216933 16847 leader_election.cc:288] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [CANDIDATE]: Term 1 election: Requested vote from peers 657d70a718194085addb9f170101a165 (127.8.151.193:38749), a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701)
I20260928 06:40:41.217257 16747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a476d2a504a47899d0cd1e2604d64fb" candidate_uuid: "19fefecd803f4948ab3dc98083a5d7fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48ef9cac41346eb997d8dc80006d7de"
I20260928 06:40:41.217268 16485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a476d2a504a47899d0cd1e2604d64fb" candidate_uuid: "19fefecd803f4948ab3dc98083a5d7fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657d70a718194085addb9f170101a165"
I20260928 06:40:41.217367 16485 raft_consensus.cc:3037] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.217362 16747 raft_consensus.cc:3037] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.218257 16747 raft_consensus.cc:2454] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19fefecd803f4948ab3dc98083a5d7fe in term 1.
I20260928 06:40:41.218338 16485 raft_consensus.cc:2454] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19fefecd803f4948ab3dc98083a5d7fe in term 1.
I20260928 06:40:41.218617 16557 leader_election.cc:302] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fefecd803f4948ab3dc98083a5d7fe, 657d70a718194085addb9f170101a165; no voters:
I20260928 06:40:41.218781 16847 raft_consensus.cc:2781] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:41.218963 16847 raft_consensus.cc:686] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 1 LEADER]: Becoming Leader. State: Replica: 19fefecd803f4948ab3dc98083a5d7fe, State: Running, Role: LEADER
I20260928 06:40:41.219228 16847 consensus_queue.cc:237] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.221196 16327 catalog_manager.cc:5495] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194). New cstate: current_term: 1 leader_uuid: "19fefecd803f4948ab3dc98083a5d7fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } health_report { overall_health: UNKNOWN } } }
Sep 28 06:40:41 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577640, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.254@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:41.272619 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.264847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38021 (local address 127.0.0.1:58150)
0928 06:40:41.265153 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:41.265165 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:41.265272 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:41.265486 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:41.265489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:41.265888 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:41.265981 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.265988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.266701 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.266704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.267268 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.267277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.267670 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.267673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.267788 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.268262 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:41.268274 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:41.269689 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:41.271292 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:41.271301 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:41.271306 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:41.271522 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:41.271757 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:41.271759 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:41.271760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:41.271841 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:41.272135 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:41.272140 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:41.272239 (+ 99us) client_negotiation.cc:769] Sending connection context
0928 06:40:41.272324 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 06:40:41.272438 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":16,"thread_start_us":111,"threads_started":1}
I20260928 06:40:41.272645 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.264869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.8.151.254:38021)
0928 06:40:41.264985 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.264989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.265236 (+ 247us) server_negotiation.cc:407] Connection header received
0928 06:40:41.265352 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.265355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.265393 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.265456 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:41.266098 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.266606 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.267370 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.267578 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.267659 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.269840 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:41.269844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:41.269846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:41.269870 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:41.271200 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:41.271619 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:41.271621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:41.271622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:41.271695 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:41.271898 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:41.271900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:41.271901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:41.272064 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:41.272131 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.272360 (+ 229us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.272521 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":44}
I20260928 06:40:41.276407 16327 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:58150:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 06:40:41.277171 16327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:41.281591 16453 tablet_service.cc:1467] Processing CreateTablet for tablet 22d4fdbdd37a4c21bf37bd2aada50595 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=667005d619504bc086cd547cd261bdfe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 06:40:41.281584 16605 tablet_service.cc:1467] Processing CreateTablet for tablet 22d4fdbdd37a4c21bf37bd2aada50595 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=667005d619504bc086cd547cd261bdfe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 06:40:41.281968 16453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22d4fdbdd37a4c21bf37bd2aada50595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.281968 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22d4fdbdd37a4c21bf37bd2aada50595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.281853 16745 tablet_service.cc:1467] Processing CreateTablet for tablet 22d4fdbdd37a4c21bf37bd2aada50595 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=667005d619504bc086cd547cd261bdfe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 06:40:41.282205 16745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22d4fdbdd37a4c21bf37bd2aada50595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.284668 16842 tablet_bootstrap.cc:492] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: Bootstrap starting.
I20260928 06:40:41.284701 16840 tablet_bootstrap.cc:492] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: Bootstrap starting.
I20260928 06:40:41.285496 16842 tablet_bootstrap.cc:654] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.285516 16840 tablet_bootstrap.cc:654] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.286090 16841 tablet_bootstrap.cc:492] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: Bootstrap starting.
I20260928 06:40:41.286746 16842 tablet_bootstrap.cc:492] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: No bootstrap required, opened a new log
I20260928 06:40:41.286767 16841 tablet_bootstrap.cc:654] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.286823 16842 ts_tablet_manager.cc:1397] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:41.287233 16842 raft_consensus.cc:348] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.287352 16842 raft_consensus.cc:374] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.287395 16842 raft_consensus.cc:729] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48ef9cac41346eb997d8dc80006d7de, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.287520 16842 consensus_queue.cc:260] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.287751 16841 tablet_bootstrap.cc:492] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: No bootstrap required, opened a new log
I20260928 06:40:41.287768 16842 ts_tablet_manager.cc:1428] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:41.287828 16841 ts_tablet_manager.cc:1397] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:41.287750 16849 tablet_replica.cc:439] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } }
I20260928 06:40:41.287904 16849 tablet_replica.cc:442] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:41.288214 16841 raft_consensus.cc:348] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.288316 16841 raft_consensus.cc:374] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.288344 16841 raft_consensus.cc:729] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657d70a718194085addb9f170101a165, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.288465 16841 consensus_queue.cc:260] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.288733 16841 ts_tablet_manager.cc:1428] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 06:40:41.288686 16848 tablet_replica.cc:439] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } }
I20260928 06:40:41.288806 16848 tablet_replica.cc:442] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:41.289294 16840 tablet_bootstrap.cc:492] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: No bootstrap required, opened a new log
I20260928 06:40:41.289374 16840 ts_tablet_manager.cc:1397] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260928 06:40:41.289806 16840 raft_consensus.cc:348] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.289911 16840 raft_consensus.cc:374] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.289942 16840 raft_consensus.cc:729] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19fefecd803f4948ab3dc98083a5d7fe, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.290052 16840 consensus_queue.cc:260] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.290299 16840 ts_tablet_manager.cc:1428] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:41.290262 16847 tablet_replica.cc:439] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } }
I20260928 06:40:41.290381 16847 tablet_replica.cc:442] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:41.475466 16848 raft_consensus.cc:482] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:41.475598 16848 raft_consensus.cc:504] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.475950 16848 leader_election.cc:288] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967), a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701)
I20260928 06:40:41.476435 16625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22d4fdbdd37a4c21bf37bd2aada50595" candidate_uuid: "657d70a718194085addb9f170101a165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19fefecd803f4948ab3dc98083a5d7fe" is_pre_election: true
I20260928 06:40:41.476442 16747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22d4fdbdd37a4c21bf37bd2aada50595" candidate_uuid: "657d70a718194085addb9f170101a165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48ef9cac41346eb997d8dc80006d7de" is_pre_election: true
I20260928 06:40:41.476593 16625 raft_consensus.cc:2454] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657d70a718194085addb9f170101a165 in term 0.
I20260928 06:40:41.476600 16747 raft_consensus.cc:2454] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657d70a718194085addb9f170101a165 in term 0.
I20260928 06:40:41.476907 16419 leader_election.cc:302] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 657d70a718194085addb9f170101a165, a48ef9cac41346eb997d8dc80006d7de; no voters:
I20260928 06:40:41.477102 16848 raft_consensus.cc:2781] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:41.477159 16848 raft_consensus.cc:482] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:41.477185 16848 raft_consensus.cc:3037] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.478125 16848 raft_consensus.cc:504] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.478426 16848 leader_election.cc:288] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [CANDIDATE]: Term 1 election: Requested vote from peers 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967), a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701)
I20260928 06:40:41.478761 16747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22d4fdbdd37a4c21bf37bd2aada50595" candidate_uuid: "657d70a718194085addb9f170101a165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48ef9cac41346eb997d8dc80006d7de"
I20260928 06:40:41.478837 16625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22d4fdbdd37a4c21bf37bd2aada50595" candidate_uuid: "657d70a718194085addb9f170101a165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19fefecd803f4948ab3dc98083a5d7fe"
I20260928 06:40:41.478874 16747 raft_consensus.cc:3037] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.478950 16625 raft_consensus.cc:3037] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.479759 16747 raft_consensus.cc:2454] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657d70a718194085addb9f170101a165 in term 1.
I20260928 06:40:41.479802 16625 raft_consensus.cc:2454] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657d70a718194085addb9f170101a165 in term 1.
I20260928 06:40:41.480026 16419 leader_election.cc:302] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 657d70a718194085addb9f170101a165, a48ef9cac41346eb997d8dc80006d7de; no voters:
I20260928 06:40:41.480175 16848 raft_consensus.cc:2781] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:41.480262 16848 raft_consensus.cc:686] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [term 1 LEADER]: Becoming Leader. State: Replica: 657d70a718194085addb9f170101a165, State: Running, Role: LEADER
I20260928 06:40:41.480393 16848 consensus_queue.cc:237] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.480995 16852 tablet_replica.cc:439] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: TxnStatusTablet state changed. Reason: New leader 657d70a718194085addb9f170101a165. Latest consensus state: current_term: 1 leader_uuid: "657d70a718194085addb9f170101a165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } }
I20260928 06:40:41.481112 16852 tablet_replica.cc:442] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: This TxnStatusTablet replica's current role is: LEADER
I20260928 06:40:41.481329 16884 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 06:40:41.481530 16884 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 06:40:41.481998 16328 catalog_manager.cc:5495] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657d70a718194085addb9f170101a165 (127.8.151.193). New cstate: current_term: 1 leader_uuid: "657d70a718194085addb9f170101a165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } health_report { overall_health: UNKNOWN } } }
I20260928 06:40:41.524657 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.521210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34200 (local address 127.8.151.193:38749)
0928 06:40:41.521321 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.521324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.521429 (+ 105us) server_negotiation.cc:407] Connection header received
0928 06:40:41.521592 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.521595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.521638 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.521700 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:41.521952 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.522455 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.523643 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.523859 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.523937 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.524251 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.524331 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.524398 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.524495 (+ 97us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.524545 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 06:40:41.524708 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.521127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38749 (local address 127.0.0.1:34200)
0928 06:40:41.521366 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:41.521380 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:41.521491 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:41.521730 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:41.521733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:41.521739 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:41.521850 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.521856 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.522540 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.522543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.523538 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.523549 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.523958 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.523961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.524046 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.524099 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.524410 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.524420 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 06:40:41.524482 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 06:40:41.524567 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260928 06:40:41.527472 16747 raft_consensus.cc:1261] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Refusing update from remote peer 657d70a718194085addb9f170101a165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:41.527535 16625 raft_consensus.cc:1261] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Refusing update from remote peer 657d70a718194085addb9f170101a165: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:41.527858 16852 consensus_queue.cc:1035] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:41.528009 16848 consensus_queue.cc:1035] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:41.529275 16849 tablet_replica.cc:439] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: TxnStatusTablet state changed. Reason: New leader 657d70a718194085addb9f170101a165. Latest consensus state: current_term: 1 leader_uuid: "657d70a718194085addb9f170101a165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } }
I20260928 06:40:41.529277 16847 tablet_replica.cc:439] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: TxnStatusTablet state changed. Reason: New leader 657d70a718194085addb9f170101a165. Latest consensus state: current_term: 1 leader_uuid: "657d70a718194085addb9f170101a165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } }
I20260928 06:40:41.529385 16849 tablet_replica.cc:442] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:41.529385 16847 tablet_replica.cc:442] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:41.531212 16852 tablet_replica.cc:439] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "657d70a718194085addb9f170101a165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } }
I20260928 06:40:41.531315 16852 tablet_replica.cc:442] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: This TxnStatusTablet replica's current role is: LEADER
I20260928 06:40:41.531612 16852 tablet_replica.cc:439] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "657d70a718194085addb9f170101a165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } }
I20260928 06:40:41.531702 16852 tablet_replica.cc:442] T 22d4fdbdd37a4c21bf37bd2aada50595 P 657d70a718194085addb9f170101a165: This TxnStatusTablet replica's current role is: LEADER
I20260928 06:40:41.531826 16889 mvcc.cc:205] Tried to move back new op lower bound from 7334206019692400640 to 7334206019505008640. Current Snapshot: MvccSnapshot[applied={T|T < 7334206019692400640}]
I20260928 06:40:41.532335 16847 tablet_replica.cc:439] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "657d70a718194085addb9f170101a165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } }
I20260928 06:40:41.532444 16847 tablet_replica.cc:442] T 22d4fdbdd37a4c21bf37bd2aada50595 P 19fefecd803f4948ab3dc98083a5d7fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:41.532997 16849 tablet_replica.cc:439] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "657d70a718194085addb9f170101a165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } } }
I20260928 06:40:41.533097 16849 tablet_replica.cc:442] T 22d4fdbdd37a4c21bf37bd2aada50595 P a48ef9cac41346eb997d8dc80006d7de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 06:40:41.542011 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.538448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49724 (local address 127.8.151.194:39967)
0928 06:40:41.538709 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.538713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.538726 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:41.538846 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.538849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.538902 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.539006 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:41.539283 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.539815 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.540642 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.540868 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.540950 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.541206 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.541641 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.541739 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.541815 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.541868 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
I20260928 06:40:41.552934 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.549686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38749 (local address 127.8.151.194:36435)
0928 06:40:41.549967 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:41.549981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:41.550079 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:41.550295 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:41.550298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:41.550305 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:41.550415 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.550421 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.551083 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.551086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.551816 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.551825 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.552259 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.552263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.552359 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.552398 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.552653 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.552659 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:40:41.552714 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 06:40:41.552789 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":98,"threads_started":1}
I20260928 06:40:41.552913 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.549792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:36435 (local address 127.8.151.193:38749)
0928 06:40:41.549983 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.549988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.550032 (+ 44us) server_negotiation.cc:407] Connection header received
0928 06:40:41.550173 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.550176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.550223 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.550280 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:41.550504 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.550989 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.551918 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.552150 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.552233 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.552514 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.552584 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.552640 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.552723 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.552774 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260928 06:40:41.559713 16747 raft_consensus.cc:1261] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Refusing update from remote peer 19fefecd803f4948ab3dc98083a5d7fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:41.559958 16485 raft_consensus.cc:1261] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Refusing update from remote peer 19fefecd803f4948ab3dc98083a5d7fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:41.560511 16847 consensus_queue.cc:1035] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:41.560704 16872 consensus_queue.cc:1035] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:41.580588 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.577118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:40255 (local address 127.8.151.194:39967)
0928 06:40:41.577253 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.577257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.577459 (+ 202us) server_negotiation.cc:407] Connection header received
0928 06:40:41.577558 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.577561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.577605 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.577676 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:41.577903 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.578493 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.579673 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.579881 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.579958 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.580235 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.580296 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.580342 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.580435 (+ 93us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.580475 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:40:41.580832 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.577040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:39967 (local address 127.8.151.193:40255)
0928 06:40:41.577383 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:41.577394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:41.577478 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:41.577695 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:41.577697 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:41.577704 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:41.577810 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.577815 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.578600 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.578603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.579564 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.579575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.579991 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.579995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.580079 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.580129 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.580361 (+ 232us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.580367 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:40:41.580418 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 06:40:41.580496 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":100,"threads_started":1}
I20260928 06:40:41.607465 16328 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:41.607698 16328 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:41.610050 16328 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5: Sending DeleteTablet for 3 replicas of tablet 4a476d2a504a47899d0cd1e2604d64fb
I20260928 06:40:41.610653 16605 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a476d2a504a47899d0cd1e2604d64fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:41 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:49722
I20260928 06:40:41.610700 16745 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a476d2a504a47899d0cd1e2604d64fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:41 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48110
I20260928 06:40:41.611052 16910 tablet_replica.cc:330] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: stopping tablet replica
I20260928 06:40:41.611220 16910 raft_consensus.cc:2229] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:41.611521 16910 raft_consensus.cc:2258] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:41.611764 16909 tablet_replica.cc:330] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: stopping tablet replica
I20260928 06:40:41.611887 16909 raft_consensus.cc:2229] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:41.611994 16909 raft_consensus.cc:2258] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:41.612613 16910 ts_tablet_manager.cc:1905] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:41.615491 16453 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a476d2a504a47899d0cd1e2604d64fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:41 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:34182
I20260928 06:40:41.615818 16909 ts_tablet_manager.cc:1905] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:41.617233 16912 tablet_replica.cc:330] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165: stopping tablet replica
I20260928 06:40:41.617398 16912 raft_consensus.cc:2229] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:41.617551 16912 raft_consensus.cc:2258] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:41.618232 16912 ts_tablet_manager.cc:1905] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:41.618728 16909 ts_tablet_manager.cc:1918] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:41.618813 16909 log.cc:1196] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/4a476d2a504a47899d0cd1e2604d64fb
I20260928 06:40:41.619076 16909 ts_tablet_manager.cc:1939] T 4a476d2a504a47899d0cd1e2604d64fb P a48ef9cac41346eb997d8dc80006d7de: Deleting consensus metadata
I20260928 06:40:41.619794 16315 catalog_manager.cc:4841] TS a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701): tablet 4a476d2a504a47899d0cd1e2604d64fb (table test-table [id=9bddb8387c88417da1bb6da088178abb]) successfully deleted
I20260928 06:40:41.620714 16910 ts_tablet_manager.cc:1918] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:41.620795 16910 log.cc:1196] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/4a476d2a504a47899d0cd1e2604d64fb
I20260928 06:40:41.621045 16910 ts_tablet_manager.cc:1939] T 4a476d2a504a47899d0cd1e2604d64fb P 19fefecd803f4948ab3dc98083a5d7fe: Deleting consensus metadata
I20260928 06:40:41.621536 16912 ts_tablet_manager.cc:1918] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:41.621659 16912 log.cc:1196] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/4a476d2a504a47899d0cd1e2604d64fb
I20260928 06:40:41.621739 16313 catalog_manager.cc:4841] TS 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967): tablet 4a476d2a504a47899d0cd1e2604d64fb (table test-table [id=9bddb8387c88417da1bb6da088178abb]) successfully deleted
I20260928 06:40:41.621995 16912 ts_tablet_manager.cc:1939] T 4a476d2a504a47899d0cd1e2604d64fb P 657d70a718194085addb9f170101a165: Deleting consensus metadata
I20260928 06:40:41.622629 16313 catalog_manager.cc:4841] TS 657d70a718194085addb9f170101a165 (127.8.151.193:38749): tablet 4a476d2a504a47899d0cd1e2604d64fb (table test-table [id=9bddb8387c88417da1bb6da088178abb]) successfully deleted
Sep 28 06:40:41 dist-test-slave-gtr8 krb5kdc[16283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 06:40:41 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:41 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 06:40:41.656499 16377 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:41.649793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.8.151.254:38021)
0928 06:40:41.649954 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.649959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.650140 (+ 181us) server_negotiation.cc:407] Connection header received
0928 06:40:41.650248 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.650252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.650298 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.650376 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:41.651211 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.651743 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.652939 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.653164 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.653253 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.656372 (+ 3119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58152: BlockingRecv error: recv got EOF from 127.0.0.1:58152 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 28 06:40:41 dist-test-slave-gtr8 krb5kdc[16283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM
I20260928 06:40:41.666082 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.657803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58156 (local address 127.8.151.254:38021)
0928 06:40:41.657905 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.657909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.658057 (+ 148us) server_negotiation.cc:407] Connection header received
0928 06:40:41.658163 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.658166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.658205 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.658266 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:41.658808 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.659337 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.660116 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.660450 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.660550 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.662827 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:41.662830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:41.662833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:41.662859 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:41.664445 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:41.664860 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:41.664863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:41.664865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:41.664944 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:41.665165 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:41.665168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:41.665169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:41.665362 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:41.665481 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.665800 (+ 319us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.665904 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 06:40:41.668493 16328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:40:41.668931 16328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:41.673503 16453 tablet_service.cc:1467] Processing CreateTablet for tablet 6495f5b0b07a4f059ee1da3f816bbbb4 (DEFAULT_TABLE table=test-table [id=013485977ca642c992c004e4a73611ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:41.673815 16453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6495f5b0b07a4f059ee1da3f816bbbb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.673753 16745 tablet_service.cc:1467] Processing CreateTablet for tablet 6495f5b0b07a4f059ee1da3f816bbbb4 (DEFAULT_TABLE table=test-table [id=013485977ca642c992c004e4a73611ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:41.673990 16745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6495f5b0b07a4f059ee1da3f816bbbb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.676282 16841 tablet_bootstrap.cc:492] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165: Bootstrap starting.
I20260928 06:40:41.676452 16605 tablet_service.cc:1467] Processing CreateTablet for tablet 6495f5b0b07a4f059ee1da3f816bbbb4 (DEFAULT_TABLE table=test-table [id=013485977ca642c992c004e4a73611ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:41.676681 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6495f5b0b07a4f059ee1da3f816bbbb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.677094 16841 tablet_bootstrap.cc:654] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.677631 16842 tablet_bootstrap.cc:492] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de: Bootstrap starting.
I20260928 06:40:41.678481 16842 tablet_bootstrap.cc:654] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.678521 16840 tablet_bootstrap.cc:492] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe: Bootstrap starting.
I20260928 06:40:41.679422 16840 tablet_bootstrap.cc:654] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.679605 16841 tablet_bootstrap.cc:492] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165: No bootstrap required, opened a new log
I20260928 06:40:41.679747 16841 ts_tablet_manager.cc:1397] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260928 06:40:41.680142 16841 raft_consensus.cc:348] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.680262 16841 raft_consensus.cc:374] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.680291 16841 raft_consensus.cc:729] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657d70a718194085addb9f170101a165, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.680413 16841 consensus_queue.cc:260] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.680661 16841 ts_tablet_manager.cc:1428] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:41.681034 16840 tablet_bootstrap.cc:492] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe: No bootstrap required, opened a new log
I20260928 06:40:41.681104 16840 ts_tablet_manager.cc:1397] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 06:40:41.681435 16842 tablet_bootstrap.cc:492] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de: No bootstrap required, opened a new log
I20260928 06:40:41.681506 16842 ts_tablet_manager.cc:1397] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260928 06:40:41.681541 16840 raft_consensus.cc:348] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.681645 16840 raft_consensus.cc:374] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.681679 16840 raft_consensus.cc:729] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19fefecd803f4948ab3dc98083a5d7fe, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.681793 16840 consensus_queue.cc:260] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.681891 16842 raft_consensus.cc:348] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.682003 16842 raft_consensus.cc:374] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.682031 16842 raft_consensus.cc:729] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48ef9cac41346eb997d8dc80006d7de, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.682034 16840 ts_tablet_manager.cc:1428] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 06:40:41.682140 16842 consensus_queue.cc:260] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.682356 16842 ts_tablet_manager.cc:1428] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:41.718636 16872 raft_consensus.cc:482] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:41.718773 16872 raft_consensus.cc:504] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.719254 16872 leader_election.cc:288] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657d70a718194085addb9f170101a165 (127.8.151.193:38749), a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701)
I20260928 06:40:41.719617 16765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6495f5b0b07a4f059ee1da3f816bbbb4" candidate_uuid: "19fefecd803f4948ab3dc98083a5d7fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48ef9cac41346eb997d8dc80006d7de" is_pre_election: true
I20260928 06:40:41.719619 16485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6495f5b0b07a4f059ee1da3f816bbbb4" candidate_uuid: "19fefecd803f4948ab3dc98083a5d7fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657d70a718194085addb9f170101a165" is_pre_election: true
I20260928 06:40:41.719771 16765 raft_consensus.cc:2454] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19fefecd803f4948ab3dc98083a5d7fe in term 0.
I20260928 06:40:41.719775 16485 raft_consensus.cc:2454] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19fefecd803f4948ab3dc98083a5d7fe in term 0.
I20260928 06:40:41.720060 16559 leader_election.cc:302] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fefecd803f4948ab3dc98083a5d7fe, a48ef9cac41346eb997d8dc80006d7de; no voters:
I20260928 06:40:41.720264 16872 raft_consensus.cc:2781] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:41.720331 16872 raft_consensus.cc:482] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:41.720376 16872 raft_consensus.cc:3037] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.721726 16872 raft_consensus.cc:504] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.722101 16872 leader_election.cc:288] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [CANDIDATE]: Term 1 election: Requested vote from peers 657d70a718194085addb9f170101a165 (127.8.151.193:38749), a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701)
I20260928 06:40:41.722441 16765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6495f5b0b07a4f059ee1da3f816bbbb4" candidate_uuid: "19fefecd803f4948ab3dc98083a5d7fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48ef9cac41346eb997d8dc80006d7de"
I20260928 06:40:41.722493 16485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6495f5b0b07a4f059ee1da3f816bbbb4" candidate_uuid: "19fefecd803f4948ab3dc98083a5d7fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657d70a718194085addb9f170101a165"
I20260928 06:40:41.722565 16765 raft_consensus.cc:3037] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.722621 16485 raft_consensus.cc:3037] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.723613 16485 raft_consensus.cc:2454] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19fefecd803f4948ab3dc98083a5d7fe in term 1.
I20260928 06:40:41.723774 16765 raft_consensus.cc:2454] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19fefecd803f4948ab3dc98083a5d7fe in term 1.
I20260928 06:40:41.723869 16557 leader_election.cc:302] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fefecd803f4948ab3dc98083a5d7fe, 657d70a718194085addb9f170101a165; no voters:
I20260928 06:40:41.724035 16872 raft_consensus.cc:2781] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:41.724120 16872 raft_consensus.cc:686] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 LEADER]: Becoming Leader. State: Replica: 19fefecd803f4948ab3dc98083a5d7fe, State: Running, Role: LEADER
I20260928 06:40:41.724258 16872 consensus_queue.cc:237] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.725870 16326 catalog_manager.cc:5495] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194). New cstate: current_term: 1 leader_uuid: "19fefecd803f4948ab3dc98083a5d7fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } health_report { overall_health: UNKNOWN } } }
I20260928 06:40:41.735759 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.732138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49730 (local address 127.8.151.194:39967)
0928 06:40:41.732265 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.732270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.732286 (+ 16us) server_negotiation.cc:407] Connection header received
0928 06:40:41.732380 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.732383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.732454 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.732541 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:41.732792 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.733549 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.734562 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.734895 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.735021 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.735294 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.735374 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.735449 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.735569 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.735625 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:40:41.737634 16485 raft_consensus.cc:1261] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Refusing update from remote peer 19fefecd803f4948ab3dc98083a5d7fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:41.738049 16765 raft_consensus.cc:1261] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Refusing update from remote peer 19fefecd803f4948ab3dc98083a5d7fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:41.738024 16872 consensus_queue.cc:1035] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:41.738473 16872 consensus_queue.cc:1035] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:41.744807 16328 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:41.744927 16328 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:41.747301 16328 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5: Sending DeleteTablet for 3 replicas of tablet 6495f5b0b07a4f059ee1da3f816bbbb4
I20260928 06:40:41.748349 16453 tablet_service.cc:1514] Processing DeleteTablet for tablet 6495f5b0b07a4f059ee1da3f816bbbb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:41 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:34182
I20260928 06:40:41.748538 16745 tablet_service.cc:1514] Processing DeleteTablet for tablet 6495f5b0b07a4f059ee1da3f816bbbb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:41 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48110
I20260928 06:40:41.748713 16605 tablet_service.cc:1514] Processing DeleteTablet for tablet 6495f5b0b07a4f059ee1da3f816bbbb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:41 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:49722
I20260928 06:40:41.748885 16604 tablet_service.cc:1514] Processing DeleteTablet for tablet 6495f5b0b07a4f059ee1da3f816bbbb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:41 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:49722
I20260928 06:40:41.748871 16328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 06:40:41.749050 16912 tablet_replica.cc:330] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165: stopping tablet replica
I20260928 06:40:41.749164 16912 raft_consensus.cc:2229] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:41.749176 16910 tablet_replica.cc:330] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe: stopping tablet replica
W20260928 06:40:41.749261 16328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:40:41.749290 16912 raft_consensus.cc:2258] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:41.749291 16910 raft_consensus.cc:2229] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:41.749451 16909 tablet_replica.cc:330] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de: stopping tablet replica
I20260928 06:40:41.749469 16910 raft_consensus.cc:2258] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:41.749549 16909 raft_consensus.cc:2229] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:41.749675 16909 raft_consensus.cc:2258] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:41.750470 16912 ts_tablet_manager.cc:1905] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:41.750715 16909 ts_tablet_manager.cc:1905] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:41.753036 16909 ts_tablet_manager.cc:1918] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:41.753105 16909 log.cc:1196] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/6495f5b0b07a4f059ee1da3f816bbbb4
I20260928 06:40:41.753115 16912 ts_tablet_manager.cc:1918] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:41.753183 16912 log.cc:1196] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/6495f5b0b07a4f059ee1da3f816bbbb4
I20260928 06:40:41.753321 16909 ts_tablet_manager.cc:1939] T 6495f5b0b07a4f059ee1da3f816bbbb4 P a48ef9cac41346eb997d8dc80006d7de: Deleting consensus metadata
I20260928 06:40:41.753418 16912 ts_tablet_manager.cc:1939] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 657d70a718194085addb9f170101a165: Deleting consensus metadata
I20260928 06:40:41.753913 16315 catalog_manager.cc:4841] TS a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701): tablet 6495f5b0b07a4f059ee1da3f816bbbb4 (table test-table [id=013485977ca642c992c004e4a73611ec]) successfully deleted
I20260928 06:40:41.754036 16313 catalog_manager.cc:4841] TS 657d70a718194085addb9f170101a165 (127.8.151.193:38749): tablet 6495f5b0b07a4f059ee1da3f816bbbb4 (table test-table [id=013485977ca642c992c004e4a73611ec]) successfully deleted
I20260928 06:40:41.754638 16910 ts_tablet_manager.cc:1905] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:41.754938 16453 tablet_service.cc:1467] Processing CreateTablet for tablet 041450a947c246cfb386aac121f01144 (DEFAULT_TABLE table=test-table [id=11e7ca0e5f0d403293efe8d9a65bfc39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:41.755206 16453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 041450a947c246cfb386aac121f01144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.756732 16910 ts_tablet_manager.cc:1918] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:40:41.756796 16910 log.cc:1196] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/6495f5b0b07a4f059ee1da3f816bbbb4
I20260928 06:40:41.757027 16910 ts_tablet_manager.cc:1939] T 6495f5b0b07a4f059ee1da3f816bbbb4 P 19fefecd803f4948ab3dc98083a5d7fe: Deleting consensus metadata
I20260928 06:40:41.757370 16841 tablet_bootstrap.cc:492] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165: Bootstrap starting.
I20260928 06:40:41.758134 16841 tablet_bootstrap.cc:654] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.758917 16745 tablet_service.cc:1467] Processing CreateTablet for tablet 041450a947c246cfb386aac121f01144 (DEFAULT_TABLE table=test-table [id=11e7ca0e5f0d403293efe8d9a65bfc39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:41.759156 16745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 041450a947c246cfb386aac121f01144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.759160 16604 tablet_service.cc:1467] Processing CreateTablet for tablet 041450a947c246cfb386aac121f01144 (DEFAULT_TABLE table=test-table [id=11e7ca0e5f0d403293efe8d9a65bfc39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:40:41.759388 16604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 041450a947c246cfb386aac121f01144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:41.760165 16313 catalog_manager.cc:4841] TS 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967): tablet 6495f5b0b07a4f059ee1da3f816bbbb4 (table test-table [id=013485977ca642c992c004e4a73611ec]) successfully deleted
W20260928 06:40:41.760421 16313 catalog_manager.cc:4808] TS 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967): delete failed for tablet 6495f5b0b07a4f059ee1da3f816bbbb4 because the tablet was not found. No further retry: Not found: Tablet not found: 6495f5b0b07a4f059ee1da3f816bbbb4
I20260928 06:40:41.760984 16841 tablet_bootstrap.cc:492] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165: No bootstrap required, opened a new log
I20260928 06:40:41.761066 16841 ts_tablet_manager.cc:1397] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260928 06:40:41.761442 16841 raft_consensus.cc:348] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.761536 16840 tablet_bootstrap.cc:492] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe: Bootstrap starting.
I20260928 06:40:41.761556 16841 raft_consensus.cc:374] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.761587 16841 raft_consensus.cc:729] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657d70a718194085addb9f170101a165, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.761693 16841 consensus_queue.cc:260] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.761992 16841 ts_tablet_manager.cc:1428] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 06:40:41.762344 16840 tablet_bootstrap.cc:654] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.763263 16842 tablet_bootstrap.cc:492] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de: Bootstrap starting.
I20260928 06:40:41.763974 16840 tablet_bootstrap.cc:492] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe: No bootstrap required, opened a new log
I20260928 06:40:41.764055 16840 ts_tablet_manager.cc:1397] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260928 06:40:41.764400 16840 raft_consensus.cc:348] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.764456 16842 tablet_bootstrap.cc:654] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de: Neither blocks nor log segments found. Creating new log.
I20260928 06:40:41.764495 16840 raft_consensus.cc:374] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.764526 16840 raft_consensus.cc:729] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19fefecd803f4948ab3dc98083a5d7fe, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.764653 16840 consensus_queue.cc:260] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.764879 16840 ts_tablet_manager.cc:1428] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:40:41.766062 16842 tablet_bootstrap.cc:492] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de: No bootstrap required, opened a new log
I20260928 06:40:41.766136 16842 ts_tablet_manager.cc:1397] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260928 06:40:41.766917 16842 raft_consensus.cc:348] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.767035 16842 raft_consensus.cc:374] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:41.767063 16842 raft_consensus.cc:729] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48ef9cac41346eb997d8dc80006d7de, State: Initialized, Role: FOLLOWER
I20260928 06:40:41.767211 16842 consensus_queue.cc:260] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.767872 16842 ts_tablet_manager.cc:1428] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260928 06:40:41.896816 16849 raft_consensus.cc:482] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:40:41.896975 16849 raft_consensus.cc:504] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.898034 16849 leader_election.cc:288] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657d70a718194085addb9f170101a165 (127.8.151.193:38749), 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967)
I20260928 06:40:41.902092 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.898431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:51377 (local address 127.8.151.193:38749)
0928 06:40:41.898598 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.898603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.898732 (+ 129us) server_negotiation.cc:407] Connection header received
0928 06:40:41.898958 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.898961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.899013 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.899084 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:41.899366 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.899899 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.901146 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.901545 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.901643 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.901694 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.901913 (+ 219us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.901979 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 06:40:41.902103 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.898796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:33485 (local address 127.8.151.194:39967)
0928 06:40:41.898917 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.898922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.899002 (+ 80us) server_negotiation.cc:407] Connection header received
0928 06:40:41.899118 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.899121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.899190 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.899265 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:40:41.899512 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.900059 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.901154 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.901549 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.901652 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.901704 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.901927 (+ 223us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.902005 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:40:41.902125 16935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.898297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38749 (local address 127.8.151.195:51377)
0928 06:40:41.898632 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:41.898646 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:41.898826 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:41.899106 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:41.899109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:41.899129 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:41.899265 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.899272 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.900018 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.900022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.901028 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.901037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.901652 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.901656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.901801 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.901818 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 06:40:41.901886 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 06:40:41.901955 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":85,"thread_start_us":89,"threads_started":1}
I20260928 06:40:41.902122 16936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.898670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:39967 (local address 127.8.151.195:33485)
0928 06:40:41.898914 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:41.898927 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:41.899028 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:41.899275 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:41.899277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:41.899291 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:40:41.899377 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.899383 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.900162 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.900165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.901039 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.901048 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.901653 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.901656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.901796 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.901815 (+ 19us) client_negotiation.cc:769] Sending connection context
0928 06:40:41.901875 (+ 60us) client_negotiation.cc:240] Negotiation successful
0928 06:40:41.901941 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":49,"thread_start_us":72,"threads_started":1}
I20260928 06:40:41.902648 16485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041450a947c246cfb386aac121f01144" candidate_uuid: "a48ef9cac41346eb997d8dc80006d7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657d70a718194085addb9f170101a165" is_pre_election: true
I20260928 06:40:41.902815 16485 raft_consensus.cc:2454] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a48ef9cac41346eb997d8dc80006d7de in term 0.
I20260928 06:40:41.902841 16625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041450a947c246cfb386aac121f01144" candidate_uuid: "a48ef9cac41346eb997d8dc80006d7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19fefecd803f4948ab3dc98083a5d7fe" is_pre_election: true
I20260928 06:40:41.902987 16625 raft_consensus.cc:2454] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a48ef9cac41346eb997d8dc80006d7de in term 0.
I20260928 06:40:41.903223 16697 leader_election.cc:302] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 657d70a718194085addb9f170101a165, a48ef9cac41346eb997d8dc80006d7de; no voters:
I20260928 06:40:41.903446 16849 raft_consensus.cc:2781] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:40:41.903508 16849 raft_consensus.cc:482] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:40:41.903553 16849 raft_consensus.cc:3037] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.904485 16849 raft_consensus.cc:504] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.904788 16849 leader_election.cc:288] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [CANDIDATE]: Term 1 election: Requested vote from peers 657d70a718194085addb9f170101a165 (127.8.151.193:38749), 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967)
I20260928 06:40:41.905107 16485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041450a947c246cfb386aac121f01144" candidate_uuid: "a48ef9cac41346eb997d8dc80006d7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657d70a718194085addb9f170101a165"
I20260928 06:40:41.905126 16625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041450a947c246cfb386aac121f01144" candidate_uuid: "a48ef9cac41346eb997d8dc80006d7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19fefecd803f4948ab3dc98083a5d7fe"
I20260928 06:40:41.905216 16625 raft_consensus.cc:3037] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.905229 16485 raft_consensus.cc:3037] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:41.906055 16625 raft_consensus.cc:2454] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a48ef9cac41346eb997d8dc80006d7de in term 1.
I20260928 06:40:41.906077 16485 raft_consensus.cc:2454] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a48ef9cac41346eb997d8dc80006d7de in term 1.
I20260928 06:40:41.906291 16697 leader_election.cc:302] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fefecd803f4948ab3dc98083a5d7fe, a48ef9cac41346eb997d8dc80006d7de; no voters:
I20260928 06:40:41.906452 16849 raft_consensus.cc:2781] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:41.906646 16849 raft_consensus.cc:686] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 1 LEADER]: Becoming Leader. State: Replica: a48ef9cac41346eb997d8dc80006d7de, State: Running, Role: LEADER
I20260928 06:40:41.906886 16849 consensus_queue.cc:237] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } }
I20260928 06:40:41.909200 16327 catalog_manager.cc:5495] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de reported cstate change: term changed from 0 to 1, leader changed from <none> to a48ef9cac41346eb997d8dc80006d7de (127.8.151.195). New cstate: current_term: 1 leader_uuid: "a48ef9cac41346eb997d8dc80006d7de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a48ef9cac41346eb997d8dc80006d7de" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 46701 } health_report { overall_health: HEALTHY } } }
I20260928 06:40:41.958065 16939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.954350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.8.151.195:46701)
0928 06:40:41.954661 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.954668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.954679 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:40:41.954746 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.954748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.954797 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.954901 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:41.955164 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.955688 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.956496 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.956793 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.956881 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.957158 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.957627 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.957728 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.957796 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.957848 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":145,"threads_started":1}
I20260928 06:40:41.967562 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.964654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:50923 (local address 127.8.151.193:38749)
0928 06:40:41.964781 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.964785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.964925 (+ 140us) server_negotiation.cc:407] Connection header received
0928 06:40:41.965024 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.965026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.965067 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.965124 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:41.965374 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.965871 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.966724 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.966925 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.966994 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.967239 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.967300 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.967349 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.967413 (+ 64us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.967451 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:40:41.967617 16942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.964566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38749 (local address 127.8.151.195:50923)
0928 06:40:41.964850 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:41.964863 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:41.964955 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:41.965180 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:41.965182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:41.965189 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:41.965287 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.965292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.965973 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.965976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.966633 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.966642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.967014 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.967017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.967085 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.967126 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.967357 (+ 231us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.967361 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 06:40:41.967399 (+ 38us) client_negotiation.cc:240] Negotiation successful
0928 06:40:41.967492 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
I20260928 06:40:41.973420 16625 raft_consensus.cc:1261] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Refusing update from remote peer a48ef9cac41346eb997d8dc80006d7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:41.973583 16485 raft_consensus.cc:1261] T 041450a947c246cfb386aac121f01144 P 657d70a718194085addb9f170101a165 [term 1 FOLLOWER]: Refusing update from remote peer a48ef9cac41346eb997d8dc80006d7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:40:41.973858 16937 consensus_queue.cc:1035] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "19fefecd803f4948ab3dc98083a5d7fe" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 39967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:41.974030 16849 consensus_queue.cc:1035] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "657d70a718194085addb9f170101a165" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:40:41.989409 16939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.986282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:51247 (local address 127.8.151.195:46701)
0928 06:40:41.986588 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:41.986591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:41.986599 (+ 8us) server_negotiation.cc:407] Connection header received
0928 06:40:41.986629 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:41.986631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:41.986673 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:41.986744 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:40:41.986978 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.987670 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.988437 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.988721 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.988799 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.989019 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.989093 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.989157 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:41.989250 (+ 93us) server_negotiation.cc:299] Negotiation successful
0928 06:40:41.989302 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260928 06:40:41.989478 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:41.986233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:46701 (local address 127.8.151.193:51247)
0928 06:40:41.986391 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:41.986405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:41.986470 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:41.986758 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:41.986760 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:41.986767 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 06:40:41.986869 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.986875 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.987746 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.987749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.988326 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:41.988334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:41.988806 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:41.988808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:41.988885 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:41.988910 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 06:40:41.989173 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 06:40:41.989180 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 06:40:41.989230 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 06:40:41.989339 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260928 06:40:42.013827 16326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:42.014016 16326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:40:42.016413 16326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ca5783a377c4667a78b224630053cb5: Sending DeleteTablet for 3 replicas of tablet 041450a947c246cfb386aac121f01144
I20260928 06:40:42.017599 16745 tablet_service.cc:1514] Processing DeleteTablet for tablet 041450a947c246cfb386aac121f01144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:42 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:48110
I20260928 06:40:42.017778 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 16400
I20260928 06:40:42.017885 16604 tablet_service.cc:1514] Processing DeleteTablet for tablet 041450a947c246cfb386aac121f01144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:40:42 UTC) from {username='oryx', principal='oryx/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:49722
I20260928 06:40:42.018412 16910 tablet_replica.cc:330] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe: stopping tablet replica
I20260928 06:40:42.018558 16910 raft_consensus.cc:2229] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:40:42.018666 16910 raft_consensus.cc:2258] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:42.018958 16909 tablet_replica.cc:330] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de: stopping tablet replica
I20260928 06:40:42.019112 16909 raft_consensus.cc:2229] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:40:42.019538 16909 raft_consensus.cc:2258] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:40:42.019793 16910 ts_tablet_manager.cc:1905] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:42.020720 16909 ts_tablet_manager.cc:1905] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:40:42.022074 16910 ts_tablet_manager.cc:1918] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:42.022150 16910 log.cc:1196] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/041450a947c246cfb386aac121f01144
I20260928 06:40:42.022384 16910 ts_tablet_manager.cc:1939] T 041450a947c246cfb386aac121f01144 P 19fefecd803f4948ab3dc98083a5d7fe: Deleting consensus metadata
I20260928 06:40:42.023196 16313 catalog_manager.cc:4841] TS 19fefecd803f4948ab3dc98083a5d7fe (127.8.151.194:39967): tablet 041450a947c246cfb386aac121f01144 (table test-table [id=11e7ca0e5f0d403293efe8d9a65bfc39]) successfully deleted
I20260928 06:40:42.025767 16909 ts_tablet_manager.cc:1918] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 06:40:42.025856 16909 log.cc:1196] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/041450a947c246cfb386aac121f01144
I20260928 06:40:42.026103 16909 ts_tablet_manager.cc:1939] T 041450a947c246cfb386aac121f01144 P a48ef9cac41346eb997d8dc80006d7de: Deleting consensus metadata
I20260928 06:40:42.026749 16315 catalog_manager.cc:4841] TS a48ef9cac41346eb997d8dc80006d7de (127.8.151.195:46701): tablet 041450a947c246cfb386aac121f01144 (table test-table [id=11e7ca0e5f0d403293efe8d9a65bfc39]) successfully deleted
W20260928 06:40:42.031648 16313 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:38749 (error 108)
I20260928 06:40:42.032220 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 16540
I20260928 06:40:42.034436 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:42.034056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38749 (local address 127.0.0.1:34214)
0928 06:40:42.034262 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:42.034336 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:38749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":58,"threads_started":1}
W20260928 06:40:42.038436 16313 catalog_manager.cc:4584] TS 657d70a718194085addb9f170101a165 (127.8.151.193:38749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 041450a947c246cfb386aac121f01144: Network error: Client connection negotiation failed: client connection to 127.8.151.193:38749: connect: Connection refused (error 111)
I20260928 06:40:42.038956 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 16680
I20260928 06:40:42.044034 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 16296
2026-09-28T06:40:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3149 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:42 dist-test-slave-gtr8 krb5kdc[16950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:42 dist-test-slave-gtr8 krb5kdc[16950](info): set up 2 sockets
Sep 28 06:40:42 dist-test-slave-gtr8 krb5kdc[16950](info): commencing operation
krb5kdc: starting...
W20260928 06:40:43.000103 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:43 dist-test-slave-gtr8 krb5kdc[16950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:43.119017 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:41587
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:41587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:43.225528 16963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:43.225808 16963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:43.225847 16963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:43.229811 16963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:43.229871 16963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:43.229887 16963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:43.229902 16963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:43.229915 16963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:43.235128 16963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:41587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:41587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:43.236146 16963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:43.237115 16963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:43.245347 16969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:43.245471 16968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:43.245437 16963 server_base.cc:1034] running on GCE node
W20260928 06:40:43.245368 16971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:43.246282 16963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:43.247426 16963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:43.248586 16963 hybrid_clock.cc:648] HybridClock initialized: now 1790577643248568 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-28T06:40:43Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1213 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:40:43 dist-test-slave-gtr8 krb5kdc[16976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:40:43 dist-test-slave-gtr8 krb5kdc[16976](info): set up 2 sockets
Sep 28 06:40:43 dist-test-slave-gtr8 krb5kdc[16976](info): commencing operation
krb5kdc: starting...
W20260928 06:40:44.224126 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:40:44 dist-test-slave-gtr8 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:40:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:40:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:40:44.341701 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:41531
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:41531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:44.446412 16989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:44.446643 16989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:44.446681 16989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:44.450538 16989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:44.450589 16989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:44.450608 16989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:44.450621 16989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:44.450634 16989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:44.455678 16989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:41531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:41531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:44.456637 16989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:44.457600 16989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:44.465778 16995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:44.465842 16994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:44.465763 16997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:44.466023 16989 server_base.cc:1034] running on GCE node
I20260928 06:40:44.466642 16989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:44.467729 16989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:44.468907 16989 hybrid_clock.cc:648] HybridClock initialized: now 1790577644468892 us; error 25 us; skew 500 ppm
Sep 28 06:40:44 dist-test-slave-gtr8 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577644, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:44.471508 16989 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:44.472350 16989 webserver.cc:460] Webserver started at http://127.8.151.254:40435/ using document root <none> and password file <none>
I20260928 06:40:44.472869 16989 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:44.472936 16989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:44.473150 16989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:44.475174 16989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "db90f6eb07b7423bb5b811274966681f"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "d1b95e095bf5e4547432b2dbb14d1b56"
server_key_iv: "99841513f08a290af71080885a4e3a6c"
server_key_version: "encryptionkey@0"
I20260928 06:40:44.475744 16989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "db90f6eb07b7423bb5b811274966681f"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "d1b95e095bf5e4547432b2dbb14d1b56"
server_key_iv: "99841513f08a290af71080885a4e3a6c"
server_key_version: "encryptionkey@0"
I20260928 06:40:44.479379 16989 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:44.481767 17004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:44.482978 16989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:44.483090 16989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "db90f6eb07b7423bb5b811274966681f"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "d1b95e095bf5e4547432b2dbb14d1b56"
server_key_iv: "99841513f08a290af71080885a4e3a6c"
server_key_version: "encryptionkey@0"
I20260928 06:40:44.483237 16989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:44.494256 16989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:44.496997 16989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:44.497154 16989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:44.503497 16989 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:41531
I20260928 06:40:44.503525 17056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:41531 every 8 connection(s)
I20260928 06:40:44.504386 16989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:44.506961 17057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:40:44.507330 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 16989
I20260928 06:40:44.507431 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:40:44.507699 8799 external_mini_cluster.cc:1428] Setting key fb93742371dfce7e5e1898f19b67317c
I20260928 06:40:44.513198 17057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: Bootstrap starting.
I20260928 06:40:44.515733 17057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: Neither blocks nor log segments found. Creating new log.
Sep 28 06:40:44 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:44.516556 17057 log.cc:824] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:44.518461 17057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: No bootstrap required, opened a new log
I20260928 06:40:44.521034 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:44.509021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44532 (local address 127.8.151.254:41531)
0928 06:40:44.509435 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:44.509449 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:44.509482 (+ 33us) server_negotiation.cc:407] Connection header received
0928 06:40:44.510140 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:44.510155 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:44.510468 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:44.510769 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:44.511582 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:44.512555 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:44.513423 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:44.514095 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:44.514259 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:44.516720 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:44.516726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:44.516743 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:44.516771 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:44.518336 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:44.518808 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:44.518813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:44.518817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:44.518927 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:44.519225 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:44.519227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:44.519228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:44.519646 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:44.519793 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:44.520064 (+ 271us) server_negotiation.cc:299] Negotiation successful
0928 06:40:44.520326 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":129,"threads_started":1}
I20260928 06:40:44.522324 17057 raft_consensus.cc:348] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } }
I20260928 06:40:44.522588 17057 raft_consensus.cc:374] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:40:44.522642 17057 raft_consensus.cc:729] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db90f6eb07b7423bb5b811274966681f, State: Initialized, Role: FOLLOWER
I20260928 06:40:44.523066 17057 consensus_queue.cc:260] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } }
I20260928 06:40:44.523237 17057 raft_consensus.cc:388] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:44.523305 17057 raft_consensus.cc:482] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:44.523399 17057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:40:44.524219 17057 raft_consensus.cc:504] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } }
I20260928 06:40:44.524518 17057 leader_election.cc:302] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db90f6eb07b7423bb5b811274966681f; no voters:
I20260928 06:40:44.524792 17057 leader_election.cc:288] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:40:44.524916 17062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:40:44.525142 17062 raft_consensus.cc:686] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 1 LEADER]: Becoming Leader. State: Replica: db90f6eb07b7423bb5b811274966681f, State: Running, Role: LEADER
I20260928 06:40:44.525424 17062 consensus_queue.cc:237] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } }
I20260928 06:40:44.525916 17057 sys_catalog.cc:564] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:44.527007 17064 sys_catalog.cc:455] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [sys.catalog]: SysCatalogTable state changed. Reason: New leader db90f6eb07b7423bb5b811274966681f. Latest consensus state: current_term: 1 leader_uuid: "db90f6eb07b7423bb5b811274966681f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } } }
I20260928 06:40:44.527007 17063 sys_catalog.cc:455] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db90f6eb07b7423bb5b811274966681f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } } }
I20260928 06:40:44.527170 17063 sys_catalog.cc:458] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:44.527170 17064 sys_catalog.cc:458] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [sys.catalog]: This master's current role is: LEADER
W20260928 06:40:44.530380 17077 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:40:44.530442 17077 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:40:44.530556 17071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:44.531520 17071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:44.536540 17071 catalog_manager.cc:1347] Generated new cluster ID: 108517ea366a41dfa533e714993a36ad
I20260928 06:40:44.536602 17071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:44.567677 17071 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:40:44.568612 17071 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:44.581702 17071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: Generated new TSK 0
I20260928 06:40:44.582399 17071 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:40:44.614034 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41531
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:44.746069 17083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:44.746310 17083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:44.746351 17083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:44.750243 17083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:44.750295 17083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:44.750349 17083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:44.755362 17083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:44.756304 17083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:44.757256 17083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:44.766139 17091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:44.766130 17088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:44.766254 17083 server_base.cc:1034] running on GCE node
W20260928 06:40:44.766142 17089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:44.766983 17083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:44.767585 17083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:44.768739 17083 hybrid_clock.cc:648] HybridClock initialized: now 1790577644768715 us; error 38 us; skew 500 ppm
Sep 28 06:40:44 dist-test-slave-gtr8 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577644, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:44.771324 17083 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:40:44.772096 17083 webserver.cc:460] Webserver started at http://127.8.151.193:41563/ using document root <none> and password file <none>
I20260928 06:40:44.772600 17083 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:44.772662 17083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:44.772876 17083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:44.774814 17083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "1ca457baaf554935b79b7c294365871b"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "ca8246cfe31e0be3d21cb0b1bc3a5239"
server_key_iv: "cf06f927aa9d05061c73a2cc49b1a449"
server_key_version: "encryptionkey@0"
I20260928 06:40:44.775370 17083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "1ca457baaf554935b79b7c294365871b"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "ca8246cfe31e0be3d21cb0b1bc3a5239"
server_key_iv: "cf06f927aa9d05061c73a2cc49b1a449"
server_key_version: "encryptionkey@0"
I20260928 06:40:44.778945 17083 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260928 06:40:44.781268 17098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:44.782457 17083 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:44.782562 17083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "1ca457baaf554935b79b7c294365871b"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "ca8246cfe31e0be3d21cb0b1bc3a5239"
server_key_iv: "cf06f927aa9d05061c73a2cc49b1a449"
server_key_version: "encryptionkey@0"
I20260928 06:40:44.782663 17083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:44.798532 17083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:44.801553 17083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:44.801721 17083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:44.802280 17083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:44.803061 17083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:44.803105 17083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:44.803189 17083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:44.803215 17083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:44.811236 17083 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:40961
I20260928 06:40:44.811273 17211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:40961 every 8 connection(s)
I20260928 06:40:44.812058 17083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:44.820685 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 17083
I20260928 06:40:44.820817 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
Sep 28 06:40:44 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577644, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:44.821094 8799 external_mini_cluster.cc:1428] Setting key e0a86ce5c93421c9f8369a9b96107813
I20260928 06:40:44.825117 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:44.813756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:54433 (local address 127.8.151.254:41531)
0928 06:40:44.813898 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:44.813902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:44.814693 (+ 791us) server_negotiation.cc:407] Connection header received
0928 06:40:44.815984 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:44.815987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:44.816043 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:44.816121 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:44.817263 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:44.817791 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:44.818892 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:44.819119 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:44.819242 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:44.821616 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:44.821620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:44.821622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:44.821648 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:44.822949 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:44.823527 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:44.823531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:44.823533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:44.823611 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:44.823977 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:44.823983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:44.823985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:44.824179 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:44.824264 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:44.824871 (+ 607us) server_negotiation.cc:299] Negotiation successful
0928 06:40:44.824985 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:40:44.826133 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:44.814114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:54433)
0928 06:40:44.814544 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:44.814593 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:44.815782 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:44.816297 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:44.816303 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:44.816646 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:44.817122 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:44.817136 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:44.817907 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:44.817910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:44.818777 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:44.818786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:44.819284 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:44.819289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:44.819448 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:44.819946 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:44.819962 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:44.821421 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:44.823074 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:44.823080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:44.823097 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:44.823438 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:44.823689 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:44.823691 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:44.823693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:44.823861 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:44.824282 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:44.824289 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:44.824598 (+ 309us) client_negotiation.cc:769] Sending connection context
0928 06:40:44.824800 (+ 202us) client_negotiation.cc:240] Negotiation successful
0928 06:40:44.825149 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":113,"threads_started":1}
I20260928 06:40:44.827409 17212 heartbeater.cc:347] Connected to a master server at 127.8.151.254:41531
I20260928 06:40:44.827678 17212 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:44.828267 17212 heartbeater.cc:511] Master 127.8.151.254:41531 requested a full tablet report, sending...
I20260928 06:40:44.829967 17021 ts_manager.cc:194] Registered new tserver with Master: 1ca457baaf554935b79b7c294365871b (127.8.151.193:40961)
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:40:44.831421 17021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:54433
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:40:44.849779 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41531
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:44.954190 17217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:44.954411 17217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:44.954448 17217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:44.958324 17217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:44.958370 17217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:44.958429 17217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:44.963402 17217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:44.964300 17217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:44.965200 17217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:44.974049 17225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:44.974049 17222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:44.974174 17223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:44.974264 17217 server_base.cc:1034] running on GCE node
I20260928 06:40:44.974747 17217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:44.975414 17217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:44.976583 17217 hybrid_clock.cc:648] HybridClock initialized: now 1790577644976557 us; error 37 us; skew 500 ppm
Sep 28 06:40:44 dist-test-slave-gtr8 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577644, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:44.979211 17217 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:40:44.980047 17217 webserver.cc:460] Webserver started at http://127.8.151.194:44877/ using document root <none> and password file <none>
I20260928 06:40:44.980554 17217 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:44.980674 17217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:44.980872 17217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:44.982914 17217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "d06c90177a514a36850dbd41ebf8ceff"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "6204ddcb764f2ce5aa667d25a6601eea"
server_key_iv: "3380b851fd1f04f70bfac5e93c08f81f"
server_key_version: "encryptionkey@0"
I20260928 06:40:44.983467 17217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "d06c90177a514a36850dbd41ebf8ceff"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "6204ddcb764f2ce5aa667d25a6601eea"
server_key_iv: "3380b851fd1f04f70bfac5e93c08f81f"
server_key_version: "encryptionkey@0"
I20260928 06:40:44.987175 17217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:44.989660 17232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:44.990914 17217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:44.991024 17217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "d06c90177a514a36850dbd41ebf8ceff"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "6204ddcb764f2ce5aa667d25a6601eea"
server_key_iv: "3380b851fd1f04f70bfac5e93c08f81f"
server_key_version: "encryptionkey@0"
I20260928 06:40:44.991119 17217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:45.001797 17217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:45.004657 17217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:45.004817 17217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:45.005348 17217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:45.006110 17217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:45.006161 17217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.006212 17217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:45.006242 17217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.014626 17217 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:45297
I20260928 06:40:45.014668 17345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:45297 every 8 connection(s)
I20260928 06:40:45.015496 17217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577644, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:45.025429 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 17217
I20260928 06:40:45.025543 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:40:45.025867 8799 external_mini_cluster.cc:1428] Setting key 482ef7e15c6506cf804c570f8c4a34c0
I20260928 06:40:45.028985 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:45.017221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:56203 (local address 127.8.151.254:41531)
0928 06:40:45.017394 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:45.017399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:45.018121 (+ 722us) server_negotiation.cc:407] Connection header received
0928 06:40:45.019400 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:45.019403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:45.019457 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:45.019528 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:45.020741 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.021265 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.022419 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.022641 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.022746 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.025028 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:45.025031 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:45.025033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:45.025056 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:45.026743 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:45.027378 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:45.027382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:45.027384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:45.027479 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:45.027884 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:45.027887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:45.027888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:45.028123 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:45.028227 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:45.028777 (+ 550us) server_negotiation.cc:299] Negotiation successful
0928 06:40:45.028872 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 06:40:45.029894 17348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:45.017554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:56203)
0928 06:40:45.017977 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:45.018019 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:45.019210 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:45.019694 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:45.019701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:45.020025 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:45.020569 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.020590 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.021393 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.021396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.022307 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.022316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.022800 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.022802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.022944 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.023463 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:45.023479 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:45.024864 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:45.026853 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:45.026858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:45.026871 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:45.027226 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:45.027575 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:45.027579 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:45.027581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:45.027781 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:45.028250 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:45.028255 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:45.028541 (+ 286us) client_negotiation.cc:769] Sending connection context
0928 06:40:45.028725 (+ 184us) client_negotiation.cc:240] Negotiation successful
0928 06:40:45.029025 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":115,"threads_started":1}
I20260928 06:40:45.030910 17346 heartbeater.cc:347] Connected to a master server at 127.8.151.254:41531
I20260928 06:40:45.031132 17346 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:45.031595 17346 heartbeater.cc:511] Master 127.8.151.254:41531 requested a full tablet report, sending...
I20260928 06:40:45.032539 17021 ts_manager.cc:194] Registered new tserver with Master: d06c90177a514a36850dbd41ebf8ceff (127.8.151.194:45297)
I20260928 06:40:45.033114 17021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:56203
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:40:45.054114 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41531
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:45.161491 17351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:45.161741 17351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:45.161783 17351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:45.165834 17351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:45.165889 17351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:45.165949 17351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:45.171167 17351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:45.172147 17351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:45.173146 17351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:45.182390 17356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:45.182397 17357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:45.182499 17359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:45.182611 17351 server_base.cc:1034] running on GCE node
I20260928 06:40:45.183173 17351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:45.183795 17351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:45.184955 17351 hybrid_clock.cc:648] HybridClock initialized: now 1790577645184936 us; error 32 us; skew 500 ppm
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577645, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:45.187799 17351 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:40:45.188669 17351 webserver.cc:460] Webserver started at http://127.8.151.195:33787/ using document root <none> and password file <none>
I20260928 06:40:45.189199 17351 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:45.189263 17351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:45.189476 17351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:40:45.191493 17351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "61c6c235cf4c4ff1bdcd38c9d357bdab"
format_stamp: "Formatted at 2026-09-28 06:40:45 on dist-test-slave-gtr8"
server_key: "3d465a09490da929e8b41067b43ceae6"
server_key_iv: "120180dfa74d1a9811b67d2de779b040"
server_key_version: "encryptionkey@0"
I20260928 06:40:45.192011 17351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "61c6c235cf4c4ff1bdcd38c9d357bdab"
format_stamp: "Formatted at 2026-09-28 06:40:45 on dist-test-slave-gtr8"
server_key: "3d465a09490da929e8b41067b43ceae6"
server_key_iv: "120180dfa74d1a9811b67d2de779b040"
server_key_version: "encryptionkey@0"
I20260928 06:40:45.195674 17351 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:45.198063 17366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.199401 17351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:40:45.199502 17351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "61c6c235cf4c4ff1bdcd38c9d357bdab"
format_stamp: "Formatted at 2026-09-28 06:40:45 on dist-test-slave-gtr8"
server_key: "3d465a09490da929e8b41067b43ceae6"
server_key_iv: "120180dfa74d1a9811b67d2de779b040"
server_key_version: "encryptionkey@0"
I20260928 06:40:45.199606 17351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:45.218881 17351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:45.221984 17351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:45.222164 17351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:45.222739 17351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:45.223541 17351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:45.223594 17351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.223651 17351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:45.223677 17351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.232081 17351 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:37593
I20260928 06:40:45.232115 17479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:37593 every 8 connection(s)
I20260928 06:40:45.232923 17351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:45.240376 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 17351
I20260928 06:40:45.240499 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:40:45.240808 8799 external_mini_cluster.cc:1428] Setting key 176c702363278303c29e3a4d9e16c0cc
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577645, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:40:45.246183 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:45.234712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:33631 (local address 127.8.151.254:41531)
0928 06:40:45.234856 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:45.234861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:45.235736 (+ 875us) server_negotiation.cc:407] Connection header received
0928 06:40:45.236981 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:45.236984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:45.237038 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:45.237123 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:45.238267 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.238812 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.240085 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.240396 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.240510 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.242954 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:45.242957 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:45.242959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:45.242983 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:45.244317 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:45.244801 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:45.244804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:45.244805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:45.244886 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:45.245178 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:45.245180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:45.245182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:45.245359 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:45.245432 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:45.245956 (+ 524us) server_negotiation.cc:299] Negotiation successful
0928 06:40:45.246066 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:40:45.246925 17482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:45.235048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:33631)
0928 06:40:45.235573 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:45.235620 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:45.236769 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:45.237257 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:45.237264 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:45.237647 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:45.238116 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.238130 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.238937 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.238940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.239936 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.239947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.240620 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.240625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.240769 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.241308 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:45.241323 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:45.242801 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:40:45.244432 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:45.244437 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:45.244449 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:45.244706 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:45.244967 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:40:45.244969 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:40:45.244971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:40:45.245105 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:40:45.245452 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:40:45.245457 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:40:45.245726 (+ 269us) client_negotiation.cc:769] Sending connection context
0928 06:40:45.245884 (+ 158us) client_negotiation.cc:240] Negotiation successful
0928 06:40:45.246128 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
I20260928 06:40:45.248153 17480 heartbeater.cc:347] Connected to a master server at 127.8.151.254:41531
I20260928 06:40:45.248394 17480 heartbeater.cc:464] Registering TS with master...
I20260928 06:40:45.248858 17480 heartbeater.cc:511] Master 127.8.151.254:41531 requested a full tablet report, sending...
I20260928 06:40:45.249840 17021 ts_manager.cc:194] Registered new tserver with Master: 61c6c235cf4c4ff1bdcd38c9d357bdab (127.8.151.195:37593)
I20260928 06:40:45.250409 17021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:33631
I20260928 06:40:45.254640 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab.
Entry for principal oryx/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab.
WARNING: no policy specified for oryx/127.8.151.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab.
Entry for principal oryx/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab.
WARNING: no policy specified for oryx/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab.
Entry for principal oryx/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab.
I20260928 06:40:45.325268 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 17083
I20260928 06:40:45.330128 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 17217
I20260928 06:40:45.334582 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 17351
I20260928 06:40:45.338786 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 16989
I20260928 06:40:45.343832 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:41531
--webserver_interface=127.8.151.254
--webserver_port=40435
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:41531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:40:45.448446 17489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:45.448701 17489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:45.448740 17489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:45.452685 17489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:40:45.452734 17489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:45.452751 17489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:45.452766 17489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:40:45.452780 17489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:40:45.457883 17489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:41531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:41531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=40435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:45.458846 17489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:45.459828 17489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:40:45.467835 17489 server_base.cc:1034] running on GCE node
W20260928 06:40:45.467878 17495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:45.467826 17494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:45.467811 17497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:45.468596 17489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:45.469579 17489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:45.470758 17489 hybrid_clock.cc:648] HybridClock initialized: now 1790577645470740 us; error 27 us; skew 500 ppm
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577645, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:45.473412 17489 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:40:45.474257 17489 webserver.cc:460] Webserver started at http://127.8.151.254:40435/ using document root <none> and password file <none>
I20260928 06:40:45.474756 17489 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:45.474825 17489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:45.478530 17489 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:45.480623 17504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.481864 17489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:40:45.481988 17489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "db90f6eb07b7423bb5b811274966681f"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "d1b95e095bf5e4547432b2dbb14d1b56"
server_key_iv: "99841513f08a290af71080885a4e3a6c"
server_key_version: "encryptionkey@0"
I20260928 06:40:45.482527 17489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:45.508419 17489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:45.511626 17489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:45.511816 17489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:45.518658 17489 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:41531
I20260928 06:40:45.518668 17556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:41531 every 8 connection(s)
I20260928 06:40:45.519747 17489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:40:45.523805 17557 sys_catalog.cc:263] Verifying existing consensus state
I20260928 06:40:45.525326 17557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: Bootstrap starting.
I20260928 06:40:45.529503 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 17489
I20260928 06:40:45.529940 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:40961
--local_ip_for_outbound_sockets=127.8.151.193
--tserver_master_addrs=127.8.151.254:41531
--webserver_port=41563
--webserver_interface=127.8.151.193
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.151.193
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:40:45.541440 17557 log.cc:824] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: Log is configured to *not* fsync() on all Append() calls
I20260928 06:40:45.547992 17557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:40:45.548322 17557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: Bootstrap complete.
I20260928 06:40:45.552371 17557 raft_consensus.cc:348] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } }
I20260928 06:40:45.552767 17557 raft_consensus.cc:729] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db90f6eb07b7423bb5b811274966681f, State: Initialized, Role: FOLLOWER
I20260928 06:40:45.553254 17557 consensus_queue.cc:260] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } }
I20260928 06:40:45.553381 17557 raft_consensus.cc:388] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:40:45.553448 17557 raft_consensus.cc:482] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:40:45.553527 17557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 1 FOLLOWER]: Advancing to term 2
I20260928 06:40:45.555442 17557 raft_consensus.cc:504] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } }
I20260928 06:40:45.555747 17557 leader_election.cc:302] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db90f6eb07b7423bb5b811274966681f; no voters:
I20260928 06:40:45.556018 17557 leader_election.cc:288] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 06:40:45.556178 17561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 2 FOLLOWER]: Leader election won for term 2
I20260928 06:40:45.556406 17561 raft_consensus.cc:686] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [term 2 LEADER]: Becoming Leader. State: Replica: db90f6eb07b7423bb5b811274966681f, State: Running, Role: LEADER
I20260928 06:40:45.556703 17561 consensus_queue.cc:237] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } }
I20260928 06:40:45.557138 17557 sys_catalog.cc:564] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:40:45.558005 17562 sys_catalog.cc:455] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "db90f6eb07b7423bb5b811274966681f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } } }
I20260928 06:40:45.558148 17562 sys_catalog.cc:458] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:45.558884 17563 sys_catalog.cc:455] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [sys.catalog]: SysCatalogTable state changed. Reason: New leader db90f6eb07b7423bb5b811274966681f. Latest consensus state: current_term: 2 leader_uuid: "db90f6eb07b7423bb5b811274966681f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db90f6eb07b7423bb5b811274966681f" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41531 } } }
I20260928 06:40:45.558990 17563 sys_catalog.cc:458] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f [sys.catalog]: This master's current role is: LEADER
I20260928 06:40:45.558938 17570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:40:45.562616 17570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:40:45.563254 17570 catalog_manager.cc:1259] Loaded cluster ID: 108517ea366a41dfa533e714993a36ad
I20260928 06:40:45.563309 17570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:40:45.565555 17570 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:40:45.566190 17570 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db90f6eb07b7423bb5b811274966681f: Loaded TSK: 0
I20260928 06:40:45.567359 17570 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 06:40:45.639358 17559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:45.639598 17559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:45.639640 17559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:45.643549 17559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:45.643610 17559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:45.643672 17559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:40:45.648805 17559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.193.keytab
--principal=oryx/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:40961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=41563
--tserver_master_addrs=127.8.151.254:41531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:45.649809 17559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:45.650785 17559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:45.661289 17585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:45.661327 17584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:45.661298 17587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:45.661695 17559 server_base.cc:1034] running on GCE node
I20260928 06:40:45.662076 17559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:45.662729 17559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:45.663908 17559 hybrid_clock.cc:648] HybridClock initialized: now 1790577645663871 us; error 53 us; skew 500 ppm
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577645, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:45.669598 17559 init.cc:377] Logged in from keytab as oryx/127.8.151.193@KRBTEST.COM (short username oryx)
I20260928 06:40:45.670490 17559 webserver.cc:460] Webserver started at http://127.8.151.193:41563/ using document root <none> and password file <none>
I20260928 06:40:45.671020 17559 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:45.671113 17559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:45.674922 17559 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260928 06:40:45.677071 17594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.678400 17559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260928 06:40:45.678539 17559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "1ca457baaf554935b79b7c294365871b"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "ca8246cfe31e0be3d21cb0b1bc3a5239"
server_key_iv: "cf06f927aa9d05061c73a2cc49b1a449"
server_key_version: "encryptionkey@0"
I20260928 06:40:45.679091 17559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:45.693807 17559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:45.696806 17559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:45.696971 17559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:45.697551 17559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:45.698343 17559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:45.698390 17559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.698443 17559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:45.698469 17559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.706732 17559 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:40961
I20260928 06:40:45.706769 17707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:40961 every 8 connection(s)
I20260928 06:40:45.707671 17559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:40:45.716012 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 17559
I20260928 06:40:45.716461 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:45297
--local_ip_for_outbound_sockets=127.8.151.194
--tserver_master_addrs=127.8.151.254:41531
--webserver_port=44877
--webserver_interface=127.8.151.194
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.151.194
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 06:40:45.722148 17710 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:45.709669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:48931 (local address 127.8.151.254:41531)
0928 06:40:45.710159 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:45.710173 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:45.710540 (+ 367us) server_negotiation.cc:407] Connection header received
0928 06:40:45.711965 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:45.711976 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:45.712273 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:45.712545 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:45.713639 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.714603 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.715778 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.716348 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.716470 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.721427 (+ 4957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:48931: BlockingRecv error: recv got EOF from 127.8.151.193:48931 (error 108)
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":131,"threads_started":1}
I20260928 06:40:45.722244 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:45.709758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:48931)
0928 06:40:45.710390 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:45.710430 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:45.711591 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:45.712679 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:45.712685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:45.713014 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:45.713452 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.713464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.714741 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.714744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.715654 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.715664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.716516 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.716520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.716766 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.717362 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:45.717383 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:45.721186 (+ 3803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":114,"threads_started":1}
W20260928 06:40:45.722882 17708 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:41531 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.151.254:41531: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:45.728259 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:45.723362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:52767)
0928 06:40:45.723477 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:45.723492 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:45.723575 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:45.723838 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:45.723841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:45.723995 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:45.724086 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.724092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.724817 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.724820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.725372 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.725380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.725795 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.725798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.725881 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.726326 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:45.726338 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:45.728129 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 06:40:45.728370 17710 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:45.723425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:52767 (local address 127.8.151.254:41531)
0928 06:40:45.723528 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:45.723532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:45.723545 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:45.723697 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:45.723699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:45.723742 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:45.723809 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:45.724201 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.724719 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.725479 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.725698 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.725784 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.728265 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:52767: BlockingRecv error: recv got EOF from 127.8.151.193:52767 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:45.733268 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:45.728899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:60405)
0928 06:40:45.728988 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:45.729001 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:45.729073 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:45.729292 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:45.729295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:45.729423 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:45.729490 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.729495 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.730167 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.730170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.730669 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.730676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.731070 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.731073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.731176 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.731565 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:45.731576 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:45.733156 (+ 1580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 06:40:45.733369 17710 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:45.728994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:60405 (local address 127.8.151.254:41531)
0928 06:40:45.729114 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:45.729118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:45.729130 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:45.729164 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:45.729167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:45.729207 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:45.729268 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:45.729589 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.730071 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.730768 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.730981 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.731061 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.733271 (+ 2210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:60405: BlockingRecv error: recv got EOF from 127.8.151.193:60405 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260928 06:40:45.733563 17708 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 06:40:45.825771 17712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:45.826028 17712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:45.826069 17712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:45.830001 17712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:45.830065 17712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:45.830123 17712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:40:45.835610 17712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.194.keytab
--principal=oryx/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:45297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=44877
--tserver_master_addrs=127.8.151.254:41531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:45.836609 17712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:45.837610 17712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:45.846896 17718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:45.846875 17717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:45.846875 17720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:45.846956 17712 server_base.cc:1034] running on GCE node
I20260928 06:40:45.847553 17712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:45.848152 17712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:45.849328 17712 hybrid_clock.cc:648] HybridClock initialized: now 1790577645849300 us; error 38 us; skew 500 ppm
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577645, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:45.851881 17712 init.cc:377] Logged in from keytab as oryx/127.8.151.194@KRBTEST.COM (short username oryx)
I20260928 06:40:45.852712 17712 webserver.cc:460] Webserver started at http://127.8.151.194:44877/ using document root <none> and password file <none>
I20260928 06:40:45.853216 17712 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:45.853281 17712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:45.857095 17712 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:40:45.859088 17727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.860296 17712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:45.860402 17712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "d06c90177a514a36850dbd41ebf8ceff"
format_stamp: "Formatted at 2026-09-28 06:40:44 on dist-test-slave-gtr8"
server_key: "6204ddcb764f2ce5aa667d25a6601eea"
server_key_iv: "3380b851fd1f04f70bfac5e93c08f81f"
server_key_version: "encryptionkey@0"
I20260928 06:40:45.860903 17712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:45.875982 17712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:45.878787 17712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:45.878944 17712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:45.879537 17712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:45.880299 17712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:45.880344 17712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.880395 17712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:45.880420 17712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:45.888741 17712 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:45297
I20260928 06:40:45.888782 17840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:45297 every 8 connection(s)
I20260928 06:40:45.889668 17712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:40:45.892941 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 17712
I20260928 06:40:45.893322 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:37593
--local_ip_for_outbound_sockets=127.8.151.195
--tserver_master_addrs=127.8.151.254:41531
--webserver_port=33787
--webserver_interface=127.8.151.195
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.151.195
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 06:40:45.900393 17710 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:45.891320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:41809 (local address 127.8.151.254:41531)
0928 06:40:45.891475 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:45.891479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:45.892294 (+ 815us) server_negotiation.cc:407] Connection header received
0928 06:40:45.893675 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:45.893678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:45.893731 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:45.893800 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:45.894912 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.895515 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.896679 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.897004 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.897113 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.900295 (+ 3182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:41809: BlockingRecv error: recv got EOF from 127.8.151.194:41809 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 06:40:45.901224 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:45.891694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:41809)
0928 06:40:45.892138 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:45.892177 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:45.893404 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:45.893976 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:45.893983 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:45.894316 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:45.894768 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.894780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.895681 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.895683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.896536 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.896548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.897139 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.897141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.897271 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.897764 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:45.897779 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:45.900194 (+ 2415us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":111,"threads_started":1}
W20260928 06:40:45.901712 17841 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:41531 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.151.254:41531: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:45.907024 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:45.902083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:46931)
0928 06:40:45.902240 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:45.902253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:45.902315 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:45.902526 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:45.902528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:45.902645 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:45.902729 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.902734 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.903775 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.903778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.904350 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.904359 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.904905 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.904908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.904990 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.905434 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:45.905445 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:45.906940 (+ 1495us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99}
W20260928 06:40:45.907227 17710 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:45.902197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:46931 (local address 127.8.151.254:41531)
0928 06:40:45.902323 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:45.902327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:45.902342 (+ 15us) server_negotiation.cc:407] Connection header received
0928 06:40:45.902383 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:45.902386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:45.902433 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:45.902520 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:45.902825 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.903646 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.904452 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.904790 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.904882 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.907078 (+ 2196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:46931: BlockingRecv error: recv got EOF from 127.8.151.194:46931 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:45 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:45.912451 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:45.907838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:38857)
0928 06:40:45.907978 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:45.907990 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:45.908048 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:45.908284 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:45.908286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:45.908412 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:45.908478 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.908483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.909462 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.909465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.909983 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:45.909991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.910483 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.910485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:45.910535 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.910929 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:45.910940 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:45.912368 (+ 1428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260928 06:40:45.912617 17710 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:45.907926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:38857 (local address 127.8.151.254:41531)
0928 06:40:45.908024 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:45.908028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:45.908040 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:45.908121 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:45.908124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:45.908163 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:45.908238 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:45.908588 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.909354 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.910075 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:45.910400 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:45.910488 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:45.912504 (+ 2016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:38857: BlockingRecv error: recv got EOF from 127.8.151.194:38857 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260928 06:40:45.912714 17841 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 06:40:46.004112 17844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:40:46.004365 17844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:40:46.004423 17844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:40:46.008364 17844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:40:46.008419 17844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:40:46.008498 17844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:40:46.013804 17844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/krb5kdc/oryx_127.8.151.195.keytab
--principal=oryx/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:37593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=33787
--tserver_master_addrs=127.8.151.254:41531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:40:46.014834 17844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:40:46.015799 17844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:40:46.025106 17852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:40:46.025110 17849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:46.025230 17844 server_base.cc:1034] running on GCE node
W20260928 06:40:46.025126 17850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:40:46.025818 17844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:40:46.026427 17844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:40:46.027655 17844 hybrid_clock.cc:648] HybridClock initialized: now 1790577646027638 us; error 31 us; skew 500 ppm
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577646, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:40:46.030171 17844 init.cc:377] Logged in from keytab as oryx/127.8.151.195@KRBTEST.COM (short username oryx)
I20260928 06:40:46.030952 17844 webserver.cc:460] Webserver started at http://127.8.151.195:33787/ using document root <none> and password file <none>
I20260928 06:40:46.031497 17844 fs_manager.cc:359] Metadata directory not provided
I20260928 06:40:46.031574 17844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:40:46.035225 17844 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260928 06:40:46.037242 17859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:46.038501 17844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:40:46.038658 17844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "61c6c235cf4c4ff1bdcd38c9d357bdab"
format_stamp: "Formatted at 2026-09-28 06:40:45 on dist-test-slave-gtr8"
server_key: "3d465a09490da929e8b41067b43ceae6"
server_key_iv: "120180dfa74d1a9811b67d2de779b040"
server_key_version: "encryptionkey@0"
I20260928 06:40:46.039216 17844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:40:46.047866 17844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:40:46.050843 17844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:40:46.051003 17844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:40:46.051610 17844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:40:46.052417 17844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:40:46.052461 17844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:46.052500 17844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:40:46.052525 17844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:40:46.061431 17844 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:37593
I20260928 06:40:46.061450 17972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:37593 every 8 connection(s)
I20260928 06:40:46.062403 17844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:40:46.069295 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 17844
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
W20260928 06:40:46.073787 17710 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:46.064163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:35987 (local address 127.8.151.254:41531)
0928 06:40:46.064334 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:46.064338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:46.065189 (+ 851us) server_negotiation.cc:407] Connection header received
0928 06:40:46.066476 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:46.066480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:46.066537 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:46.066608 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:46.067784 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.068390 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.069787 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.070084 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.070177 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.073660 (+ 3483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:35987: BlockingRecv error: recv got EOF from 127.8.151.195:35987 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:40:46.074388 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:46.064561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:35987)
0928 06:40:46.065052 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:46.065094 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:46.066275 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:46.066792 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:46.066800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:46.067165 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:46.067646 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:46.067659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.068509 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.068512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:46.069678 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:46.069688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.070323 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.070326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:46.070499 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.071189 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:46.071209 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:46.073545 (+ 2336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":146,"threads_started":1}
W20260928 06:40:46.074820 17973 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:41531 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.151.254:41531: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
I20260928 06:40:46.078531 17977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:46.070263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.8.151.254:41531)
0928 06:40:46.070512 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:46.070516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:46.070528 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:46.070559 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:46.070561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:46.070602 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:46.070694 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:46.071549 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.072143 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.073213 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.073430 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.073512 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.074808 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:40:46.074814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:40:46.074827 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:40:46.074861 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:40:46.076597 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:46.076980 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:46.076985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:46.076989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:46.077101 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:40:46.077367 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:40:46.077371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:40:46.077373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:40:46.077748 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:40:46.077936 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0928 06:40:46.078190 (+ 254us) server_negotiation.cc:299] Negotiation successful
0928 06:40:46.078418 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:46.079964 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:46.075280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:38851)
0928 06:40:46.075370 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:46.075383 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:46.075446 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:46.075639 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:46.075641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:46.075770 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:46.075850 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:46.075855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.076508 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.076511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:46.077025 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:46.077032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.077432 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.077436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:46.077522 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.078139 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:46.078153 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:46.079848 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 06:40:46.080024 17710 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:46.075347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:38851 (local address 127.8.151.254:41531)
0928 06:40:46.075451 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:46.075454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:46.075482 (+ 28us) server_negotiation.cc:407] Connection header received
0928 06:40:46.075509 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:46.075511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:46.075541 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:46.075601 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:46.075949 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.076416 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.077131 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.077343 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.077421 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.079952 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:38851: BlockingRecv error: recv got EOF from 127.8.151.195:38851 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:46.084988 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:46.080480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:46849)
0928 06:40:46.080650 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:46.080663 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:46.080732 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:46.081024 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:46.081027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:46.081192 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:46.081282 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:46.081288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.081996 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.081999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:46.082566 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:46.082576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.082976 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.082979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:46.083050 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.083434 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:46.083445 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:46.084904 (+ 1459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104}
W20260928 06:40:46.085122 17710 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:46.080707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:46849 (local address 127.8.151.254:41531)
0928 06:40:46.080836 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:46.080840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:46.080850 (+ 10us) server_negotiation.cc:407] Connection header received
0928 06:40:46.080879 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:46.080881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:46.080919 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:46.080986 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:46.081423 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.081879 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.082683 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.082884 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.082951 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.085022 (+ 2071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:46849: BlockingRecv error: recv got EOF from 127.8.151.195:46849 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260928 06:40:46.085170 17973 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:46.739545 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:46.734203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:46949)
0928 06:40:46.734503 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:46.734517 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:46.734616 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:46.734893 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:46.734895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:46.735060 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:46.735195 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:46.735201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.735921 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.735924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:46.736479 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:46.736486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.736974 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.736977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:46.737045 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.737544 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:46.737556 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:46.739386 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":73,"threads_started":1}
W20260928 06:40:46.739642 17979 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:46.734279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:46949 (local address 127.8.151.254:41531)
0928 06:40:46.734551 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:46.734554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:46.734568 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:40:46.734734 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:46.734737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:46.734786 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:46.734868 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:46.735300 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.735834 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.736596 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.736846 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.736933 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.739521 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:46949: BlockingRecv error: recv got EOF from 127.8.151.193:46949 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":101,"threads_started":1}
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:46 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:46.918427 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:46.913313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:41635)
0928 06:40:46.913578 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:46.913601 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:46.913707 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:46.913938 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:46.913940 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:46.914108 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:46.914204 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:46.914209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.914899 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.914901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:46.915551 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:46.915560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.915959 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.915962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:46.916032 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.916472 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:46.916484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:46.918279 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":77,"threads_started":1}
W20260928 06:40:46.918522 17979 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:46.913379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:41635 (local address 127.8.151.254:41531)
0928 06:40:46.913523 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:46.913528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:46.913668 (+ 140us) server_negotiation.cc:407] Connection header received
0928 06:40:46.913798 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:46.913800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:46.913847 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:46.913907 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:46.914287 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.914790 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.915645 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:46.915847 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:46.915910 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:46.918424 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:41635: BlockingRecv error: recv got EOF from 127.8.151.194:41635 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 28 06:40:47 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:47 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:47.090989 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:47.085778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:35527)
0928 06:40:47.086126 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:47.086148 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:47.086246 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:47.086497 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:47.086500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:47.086706 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:47.086849 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:47.086858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.087639 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.087642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:47.088234 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:47.088242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.088678 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.088680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:47.088744 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:47.089176 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:47.089186 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:47.090872 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":100,"threads_started":1}
W20260928 06:40:47.091099 17979 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:47.085839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:35527 (local address 127.8.151.254:41531)
0928 06:40:47.085976 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:47.085980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:47.086239 (+ 259us) server_negotiation.cc:407] Connection header received
0928 06:40:47.086327 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:47.086331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:47.086383 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:47.086465 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:47.086963 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.087541 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.088355 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.088590 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.088676 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:47.090998 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:35527: BlockingRecv error: recv got EOF from 127.8.151.195:35527 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 28 06:40:47 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:47 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:47.745668 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:47.740316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:50633)
0928 06:40:47.740612 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:47.740633 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:47.740737 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:47.740951 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:47.740953 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:47.741131 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:47.741229 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:47.741235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.742054 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.742057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:47.742611 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:47.742619 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.743107 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.743110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:47.743218 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:47.743712 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:47.743723 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:47.745494 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":101,"threads_started":1}
W20260928 06:40:47.745775 17983 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:47.740391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:50633 (local address 127.8.151.254:41531)
0928 06:40:47.740618 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:47.740622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:47.740729 (+ 107us) server_negotiation.cc:407] Connection header received
0928 06:40:47.740802 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:47.740804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:47.740848 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:47.740922 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:47.741352 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.741942 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.742755 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.742997 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.743076 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:47.745644 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:50633: BlockingRecv error: recv got EOF from 127.8.151.193:50633 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":54,"threads_started":1}
Sep 28 06:40:47 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:47 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:47.924506 17984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:47.919297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:38855)
0928 06:40:47.919550 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:47.919566 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:47.919664 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:47.919922 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:47.919925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:47.920094 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:47.920191 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:47.920197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.920906 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.920910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:47.921492 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:47.921501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.921983 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.921985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:47.922050 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:47.922497 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:47.922508 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:47.924366 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":79,"threads_started":1}
W20260928 06:40:47.924676 17983 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:47.919336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:38855 (local address 127.8.151.254:41531)
0928 06:40:47.919467 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:47.919472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:47.919634 (+ 162us) server_negotiation.cc:407] Connection header received
0928 06:40:47.919744 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:47.919746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:47.919799 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:47.919876 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:47.920284 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.920794 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.921615 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:47.921848 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:47.921926 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:47.924532 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:38855: BlockingRecv error: recv got EOF from 127.8.151.194:38855 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 28 06:40:48 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:48 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:48.096881 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:48.091734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:54433)
0928 06:40:48.092007 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:48.092021 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:48.092126 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:48.092367 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:48.092370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:48.092556 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:48.092658 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:48.092664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.093389 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.093392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:48.093986 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:48.093995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.094418 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.094422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:48.094500 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:48.094977 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:48.094988 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:48.096732 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":98,"threads_started":1}
W20260928 06:40:48.096973 17983 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:48.091869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:54433 (local address 127.8.151.254:41531)
0928 06:40:48.092002 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:48.092006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:48.092072 (+ 66us) server_negotiation.cc:407] Connection header received
0928 06:40:48.092219 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:48.092222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:48.092269 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:48.092333 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:48.092762 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.093289 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.094095 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.094320 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.094404 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:48.096872 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:54433: BlockingRecv error: recv got EOF from 127.8.151.195:54433 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 28 06:40:48 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:48 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:48.751435 17986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:48.746364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:35359)
0928 06:40:48.746612 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:48.746625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:48.746733 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:48.746949 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:48.746951 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:48.747123 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:48.747272 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:48.747279 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.748032 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.748035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:48.748610 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:48.748620 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.749062 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.749065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:48.749131 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:48.749604 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:48.749615 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:48.751285 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
W20260928 06:40:48.751531 17987 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:48.746411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:35359 (local address 127.8.151.254:41531)
0928 06:40:48.746633 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:48.746636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:48.746680 (+ 44us) server_negotiation.cc:407] Connection header received
0928 06:40:48.746805 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:48.746808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:48.746850 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:48.746924 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:48.747368 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.747911 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.748730 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.748952 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.749027 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:48.751417 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:35359: BlockingRecv error: recv got EOF from 127.8.151.193:35359 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
Sep 28 06:40:48 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:48 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:48.930521 17988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:48.925341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:40047)
0928 06:40:48.925596 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:48.925609 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:48.925709 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:48.925986 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:48.925988 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:48.926155 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:48.926264 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:48.926269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.926979 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.926981 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:48.927587 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:48.927595 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.928051 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.928054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:48.928118 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:48.928614 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:48.928626 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:48.930403 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":72,"threads_started":1}
W20260928 06:40:48.930660 17987 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:48.925451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:40047 (local address 127.8.151.254:41531)
0928 06:40:48.925598 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:48.925602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:48.925695 (+ 93us) server_negotiation.cc:407] Connection header received
0928 06:40:48.925833 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:48.925836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:48.925885 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:48.925961 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:48.926363 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.926879 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.927687 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:48.927938 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:48.928015 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:48.930547 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:40047: BlockingRecv error: recv got EOF from 127.8.151.194:40047 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 28 06:40:49 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:49 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:49.102855 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:49.097635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:43361)
0928 06:40:49.097954 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:49.097966 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:49.098076 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:49.098328 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:49.098331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:49.098521 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:49.098614 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:49.098620 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.099396 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.099400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:49.099992 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:49.100000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.100429 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.100432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:49.100511 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:49.100969 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:49.100980 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:49.102646 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":124,"threads_started":1}
W20260928 06:40:49.102887 17987 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:49.097745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:43361 (local address 127.8.151.254:41531)
0928 06:40:49.097884 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:49.097889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:49.098036 (+ 147us) server_negotiation.cc:407] Connection header received
0928 06:40:49.098163 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:49.098166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:49.098215 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:49.098293 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:49.098728 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.099298 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.100101 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.100329 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.100413 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:49.102786 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:43361: BlockingRecv error: recv got EOF from 127.8.151.195:43361 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 28 06:40:49 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:49 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:49.757405 17990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:49.752251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:58199)
0928 06:40:49.752516 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:49.752533 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:49.752624 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:49.752880 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:49.752882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:49.753047 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:49.753139 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:49.753144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.753874 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.753877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:49.754473 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:49.754480 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.754917 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.754919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:49.754977 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:49.755454 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:49.755465 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:49.757293 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":72,"threads_started":1}
W20260928 06:40:49.757541 17991 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:49.752371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:58199 (local address 127.8.151.254:41531)
0928 06:40:49.752608 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:49.752612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:49.752625 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:40:49.752704 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:49.752706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:49.752786 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:49.752852 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:49.753220 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.753760 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.754582 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.754830 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.754908 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:49.757411 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:58199: BlockingRecv error: recv got EOF from 127.8.151.193:58199 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":76,"threads_started":1}
Sep 28 06:40:49 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:49 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:49.936399 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:49.931266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:36393)
0928 06:40:49.931566 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:49.931578 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:49.931676 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:49.931894 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:49.931896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:49.932066 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:49.932162 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:49.932167 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.932958 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.932961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:49.933541 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:49.933551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.933917 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.933919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:49.933973 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:49.934459 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:49.934470 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:49.936233 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":103,"threads_started":1}
W20260928 06:40:49.936479 17991 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:49.931297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:36393 (local address 127.8.151.254:41531)
0928 06:40:49.931424 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:49.931427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:49.931683 (+ 256us) server_negotiation.cc:407] Connection header received
0928 06:40:49.931736 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:49.931738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:49.931787 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:49.931862 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:49.932306 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.932843 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.933632 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:49.933835 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:49.933905 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:49.936380 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:36393: BlockingRecv error: recv got EOF from 127.8.151.194:36393 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 28 06:40:50 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:50 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:50.108616 17993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:50.103572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:58549)
0928 06:40:50.103808 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:50.103820 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:50.103913 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:50.104117 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:50.104119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:50.104282 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:50.104389 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:50.104395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.105087 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.105089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:50.105641 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:50.105648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.106084 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.106086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:50.106145 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:50.106627 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:50.106642 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:50.108498 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":69,"threads_started":1}
W20260928 06:40:50.108770 17991 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:50.103649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:58549 (local address 127.8.151.254:41531)
0928 06:40:50.103771 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:50.103775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:50.103879 (+ 104us) server_negotiation.cc:407] Connection header received
0928 06:40:50.103981 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:50.103983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:50.104027 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:50.104093 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:50.104489 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.105001 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.105767 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.105988 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.106071 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:50.108646 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:58549: BlockingRecv error: recv got EOF from 127.8.151.195:58549 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 28 06:40:50 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:50 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:50.767880 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:50.758270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:38707)
0928 06:40:50.758554 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:50.758568 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:50.758669 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:50.759046 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:50.759049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:50.759275 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:50.759389 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:50.759395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.760219 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.760222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:50.760802 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:50.760811 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.761236 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.761239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:50.761311 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:50.765988 (+ 4677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:50.766002 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:50.767757 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
W20260928 06:40:50.768069 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:50.758350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:38707 (local address 127.8.151.254:41531)
0928 06:40:50.758616 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:50.758620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:50.758634 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:40:50.758796 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:50.758799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:50.758882 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:50.758970 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:50.759521 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.760100 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.760896 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.761109 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.761177 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:50.767930 (+ 6753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:38707: BlockingRecv error: recv got EOF from 127.8.151.193:38707 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":90,"threads_started":1}
Sep 28 06:40:50 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:50 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:50.942804 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:50.937280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:33337)
0928 06:40:50.937598 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:50.937612 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:50.937711 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:50.937955 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:50.937957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:50.938132 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:50.938224 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:50.938229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.939267 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.939271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:50.940049 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:50.940058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.940471 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.940474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:50.940535 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:50.941029 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:50.941045 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:50.942679 (+ 1634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":105,"threads_started":1}
W20260928 06:40:50.942965 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:50.937493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:33337 (local address 127.8.151.254:41531)
0928 06:40:50.937625 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:50.937628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:50.937665 (+ 37us) server_negotiation.cc:407] Connection header received
0928 06:40:50.937791 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:50.937794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:50.937835 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:50.937900 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:50.938361 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.939118 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.940155 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:50.940371 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:50.940448 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:50.942832 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:33337: BlockingRecv error: recv got EOF from 127.8.151.194:33337 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 28 06:40:51 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:51 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:51.114626 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:51.109351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:34535)
0928 06:40:51.109695 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:51.109707 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:51.109833 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:51.110047 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:51.110049 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:51.110214 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:51.110313 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:51.110318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.111000 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.111002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:51.111581 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:51.111588 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.111989 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.111992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:51.112075 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:51.112566 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:51.112578 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:51.114502 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":84,"threads_started":1}
W20260928 06:40:51.114727 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:51.109440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:34535 (local address 127.8.151.254:41531)
0928 06:40:51.109573 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:51.109577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:51.109763 (+ 186us) server_negotiation.cc:407] Connection header received
0928 06:40:51.109909 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:51.109911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:51.109955 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:51.110024 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:51.110408 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.110919 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.111661 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.111896 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.111988 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:51.114629 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:34535: BlockingRecv error: recv got EOF from 127.8.151.195:34535 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 28 06:40:51 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:51 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:51.773823 17998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:51.768600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:47057)
0928 06:40:51.768872 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:51.768905 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:51.768992 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:51.769213 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:51.769215 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:51.769403 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:51.769503 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:51.769508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.770313 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.770316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:51.770914 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:51.770924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.771392 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.771395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:51.771459 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:51.771958 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:51.771969 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:51.773708 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":80,"threads_started":1}
W20260928 06:40:51.773978 17999 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:51.768689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:47057 (local address 127.8.151.254:41531)
0928 06:40:51.768924 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:51.768927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:51.768950 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:40:51.769068 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:51.769070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:51.769114 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:51.769190 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:51.769606 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.770195 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.771040 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.771290 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.771375 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:51.773852 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:47057: BlockingRecv error: recv got EOF from 127.8.151.193:47057 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":58,"threads_started":1}
Sep 28 06:40:51 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:51 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:51.948575 18000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:51.943498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:52839)
0928 06:40:51.943754 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:51.943770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:51.943889 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:51.944140 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:51.944143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:51.944343 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:51.944434 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:51.944440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.945165 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.945169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:51.945722 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:51.945729 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.946150 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.946154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:51.946229 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:51.946676 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:51.946687 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:51.948427 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":77,"threads_started":1}
W20260928 06:40:51.948657 17999 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:51.943578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:52839 (local address 127.8.151.254:41531)
0928 06:40:51.943735 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:51.943740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:51.943844 (+ 104us) server_negotiation.cc:407] Connection header received
0928 06:40:51.943991 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:51.943994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:51.944040 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:51.944103 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:51.944543 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.945067 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.945830 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:51.946046 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:51.946136 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:51.948558 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:52839: BlockingRecv error: recv got EOF from 127.8.151.194:52839 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 28 06:40:52 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:52 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:52.120412 18001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:52.115405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:41759)
0928 06:40:52.115670 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:52.115685 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:52.115798 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:52.116043 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:52.116046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:52.116233 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:52.116323 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:52.116329 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.117066 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.117069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:52.117610 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:52.117618 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.118034 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.118037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:52.118113 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:52.118594 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:52.118606 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:52.120294 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":79,"threads_started":1}
W20260928 06:40:52.120687 17999 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:52.115523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:41759 (local address 127.8.151.254:41531)
0928 06:40:52.115671 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:52.115676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:52.115751 (+ 75us) server_negotiation.cc:407] Connection header received
0928 06:40:52.115885 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:52.115889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:52.115940 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:52.116007 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:52.116453 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.116968 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.117721 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.117940 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.118021 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:52.120464 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:41759: BlockingRecv error: recv got EOF from 127.8.151.195:41759 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 28 06:40:52 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:52 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:52.780089 18002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:52.774603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:46375)
0928 06:40:52.774943 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:52.774956 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:52.775052 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:52.775285 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:52.775287 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:52.775475 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:52.775566 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:52.775571 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.776440 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.776444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:52.777265 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:52.777277 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.777695 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.777698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:52.777760 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:52.778249 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:52.778261 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:52.779971 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":72,"threads_started":1}
W20260928 06:40:52.780208 18003 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:52.774710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:46375 (local address 127.8.151.254:41531)
0928 06:40:52.774936 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:52.774939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:52.775008 (+ 69us) server_negotiation.cc:407] Connection header received
0928 06:40:52.775117 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:52.775120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:52.775180 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:52.775258 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:52.775652 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.776299 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.777376 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.777599 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.777677 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:52.780096 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:46375: BlockingRecv error: recv got EOF from 127.8.151.193:46375 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":63,"threads_started":1}
Sep 28 06:40:52 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:52 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:52.954588 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:52.949438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:51705)
0928 06:40:52.949803 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:52.949819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:52.949929 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:52.950192 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:52.950194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:52.950375 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:52.950468 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:52.950474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.951327 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.951330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:52.951881 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:52.951890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.952327 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.952329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:52.952387 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:52.952818 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:52.952828 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:52.954471 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":89,"threads_started":1}
W20260928 06:40:52.954702 18003 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:52.949509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:51705 (local address 127.8.151.254:41531)
0928 06:40:52.949647 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:52.949651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:52.949895 (+ 244us) server_negotiation.cc:407] Connection header received
0928 06:40:52.950006 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:52.950008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:52.950053 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:52.950118 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:52.950609 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.951203 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.952010 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:52.952235 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:52.952318 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:52.954600 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:51705: BlockingRecv error: recv got EOF from 127.8.151.194:51705 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 28 06:40:53 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:53 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:53.126529 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:53.121271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:51629)
0928 06:40:53.121574 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:53.121586 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:53.121686 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:53.121901 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:53.121903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:53.122070 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:53.122170 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:53.122175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.122907 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.122911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:53.123701 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:53.123713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.124141 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.124143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:53.124205 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:53.124683 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:53.124698 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:53.126364 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":79,"threads_started":1}
W20260928 06:40:53.126600 18003 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:53.121400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:51629 (local address 127.8.151.254:41531)
0928 06:40:53.121525 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:53.121528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:53.121648 (+ 120us) server_negotiation.cc:407] Connection header received
0928 06:40:53.121756 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:53.121758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:53.121804 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:53.121873 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:53.122252 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.122764 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.123818 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.124052 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.124125 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:53.126500 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:51629: BlockingRecv error: recv got EOF from 127.8.151.195:51629 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 28 06:40:53 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:53 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:53.785969 18006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:53.780770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:59779)
0928 06:40:53.781098 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:53.781111 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:53.781196 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:53.781428 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:53.781431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:53.781599 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:53.781692 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:53.781697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.782513 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.782517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:53.783093 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:53.783125 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.783563 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.783566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:53.783629 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:53.784081 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:53.784092 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:53.785812 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":82,"threads_started":1}
W20260928 06:40:53.786087 18007 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:53.780956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:59779 (local address 127.8.151.254:41531)
0928 06:40:53.781164 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:53.781167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:53.781179 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:53.781261 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:53.781263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:53.781313 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:53.781385 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:53.781771 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.782412 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.783236 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.783473 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.783553 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:53.785935 (+ 2382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:59779: BlockingRecv error: recv got EOF from 127.8.151.193:59779 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":61,"threads_started":1}
Sep 28 06:40:53 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:53 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:53.960903 18008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:53.955324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:32875)
0928 06:40:53.955658 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:53.955673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:53.955784 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:53.956018 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:53.956021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:53.956210 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:53.956302 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:53.956308 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.957194 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.957198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:53.957843 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:53.957852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.958308 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.958312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:53.958412 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:53.958894 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:53.958908 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:53.960755 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":74,"threads_started":1}
W20260928 06:40:53.961165 18007 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:53.955428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:32875 (local address 127.8.151.254:41531)
0928 06:40:53.955573 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:53.955577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:53.955728 (+ 151us) server_negotiation.cc:407] Connection header received
0928 06:40:53.955868 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:53.955871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:53.955920 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:53.955982 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:53.956411 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.957092 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.957950 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:53.958174 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:53.958278 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:53.960892 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:32875: BlockingRecv error: recv got EOF from 127.8.151.194:32875 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 28 06:40:54 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:54 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:54.133510 18009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:54.127282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:40227)
0928 06:40:54.127588 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:54.127601 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:54.127699 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:54.127960 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:54.127962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:54.128126 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:54.128224 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:54.128228 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.129253 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.129255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:54.129913 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:54.129924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.130514 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.130518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:54.130602 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:54.131165 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:54.131181 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:54.133373 (+ 2192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":78,"threads_started":1}
W20260928 06:40:54.133689 18007 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:54.127387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:40227 (local address 127.8.151.254:41531)
0928 06:40:54.127546 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:54.127552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:54.127670 (+ 118us) server_negotiation.cc:407] Connection header received
0928 06:40:54.127775 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:54.127778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:54.127838 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:54.127946 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:54.128335 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.129157 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.130039 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.130395 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.130503 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:54.133559 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:40227: BlockingRecv error: recv got EOF from 127.8.151.195:40227 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 28 06:40:54 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:54 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:54.792315 18010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:54.786693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:34719)
0928 06:40:54.786951 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:54.786968 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:54.787073 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:54.787343 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:54.787346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:54.787542 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:54.787638 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:54.787644 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.788427 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.788429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:54.788985 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:54.788994 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.789416 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.789419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:54.789505 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:54.790034 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:54.790045 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:54.792139 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
W20260928 06:40:54.792698 18011 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:54.786760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:34719 (local address 127.8.151.254:41531)
0928 06:40:54.787061 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:54.787066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:54.787080 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:40:54.787178 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:54.787182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:54.787229 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:54.787304 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:54.787748 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.788297 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.789091 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.789316 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.789399 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:54.792449 (+ 3050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:34719: BlockingRecv error: recv got EOF from 127.8.151.193:34719 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":98,"threads_started":1}
Sep 28 06:40:54 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:54 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:54.967157 18012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:54.961643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:40489)
0928 06:40:54.961949 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:54.961961 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:54.962083 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:54.962332 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:54.962335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:54.962523 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:54.962613 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:54.962620 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.963385 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.963388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:54.963955 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:54.963964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.964387 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.964390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:54.964465 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:54.964922 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:54.964938 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:54.966952 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":78,"threads_started":1}
W20260928 06:40:54.967710 18011 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:54.961698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:40489 (local address 127.8.151.254:41531)
0928 06:40:54.961821 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:54.961825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:54.962047 (+ 222us) server_negotiation.cc:407] Connection header received
0928 06:40:54.962167 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:54.962169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:54.962223 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:54.962292 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:54.962720 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.963289 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.964074 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:54.964293 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:54.964376 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:54.967609 (+ 3233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:40489: BlockingRecv error: recv got EOF from 127.8.151.194:40489 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 28 06:40:55 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:55 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:55.139706 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:55.134306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:45211)
0928 06:40:55.134581 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:55.134594 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:55.134705 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:55.134949 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:55.134952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:55.135192 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:55.135299 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:55.135306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.136054 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.136058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:55.136624 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:55.136631 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.137057 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.137061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:55.137139 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:55.137666 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:55.137677 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:55.139590 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
W20260928 06:40:55.139875 18011 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:55.134369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:45211 (local address 127.8.151.254:41531)
0928 06:40:55.134500 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:55.134504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:55.134663 (+ 159us) server_negotiation.cc:407] Connection header received
0928 06:40:55.134795 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:55.134798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:55.134846 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:55.134911 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:55.135411 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.135944 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.136735 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.136951 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.137044 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:55.139757 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:45211: BlockingRecv error: recv got EOF from 127.8.151.195:45211 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 28 06:40:55 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:55 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:55.802821 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:55.793109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:47227)
0928 06:40:55.793354 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:55.793366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:55.793445 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:55.793717 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:55.793719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:55.793885 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:55.793976 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:55.793981 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.794984 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.794986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:55.795583 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:55.795591 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.796137 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.796139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:55.796197 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:55.800646 (+ 4449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:55.800661 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:55.802705 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
W20260928 06:40:55.803033 18015 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:55.793207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:47227 (local address 127.8.151.254:41531)
0928 06:40:55.793475 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:55.793481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:55.793497 (+ 16us) server_negotiation.cc:407] Connection header received
0928 06:40:55.793545 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:55.793548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:55.793601 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:55.793710 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:55.794087 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.794886 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.795685 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.796031 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.796144 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:55.802882 (+ 6738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:47227: BlockingRecv error: recv got EOF from 127.8.151.193:47227 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":85,"threads_started":1}
Sep 28 06:40:55 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:55 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:55.974668 18016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:55.968026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:40395)
0928 06:40:55.968307 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:55.968318 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:55.968425 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:55.968657 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:55.968659 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:55.968861 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:55.968950 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:55.968956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.969985 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.969988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:55.970549 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:55.970558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.971104 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.971109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:55.971247 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:55.971826 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:55.971841 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:55.974497 (+ 2656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":107,"threads_started":1}
W20260928 06:40:55.975311 18015 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:55.968089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:40395 (local address 127.8.151.254:41531)
0928 06:40:55.968226 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:55.968231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:55.968389 (+ 158us) server_negotiation.cc:407] Connection header received
0928 06:40:55.968512 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:55.968515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:55.968561 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:55.968623 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:55.969077 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.969882 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.970672 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:55.970972 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:55.971071 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:55.975225 (+ 4154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:40395: BlockingRecv error: recv got EOF from 127.8.151.194:40395 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 28 06:40:56 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:56 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:56.146212 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:56.140605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:33611)
0928 06:40:56.140942 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:56.140957 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:56.141061 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:56.141301 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:56.141304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:56.141498 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:56.141588 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:56.141593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.142387 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.142390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:56.142946 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:56.142954 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.143493 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.143496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:56.143577 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:56.144043 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:56.144056 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:56.146054 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":102,"threads_started":1}
W20260928 06:40:56.146286 18015 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:56.140775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:33611 (local address 127.8.151.254:41531)
0928 06:40:56.140898 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:56.140903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:56.141024 (+ 121us) server_negotiation.cc:407] Connection header received
0928 06:40:56.141159 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:56.141162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:56.141210 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:56.141269 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:56.141697 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.142285 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.143054 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.143378 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.143470 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:56.146186 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:33611: BlockingRecv error: recv got EOF from 127.8.151.195:33611 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 28 06:40:56 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:56 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:56.809744 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:56.803715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:33761)
0928 06:40:56.804098 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:56.804113 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:56.804353 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:56.804651 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:56.804654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:56.804840 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:56.804948 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:56.804954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.805956 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.805958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:56.806494 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:56.806500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.807046 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.807049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:56.807112 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:56.807692 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:56.807704 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:56.809630 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":99,"threads_started":1}
W20260928 06:40:56.809912 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:56.803857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:33761 (local address 127.8.151.254:41531)
0928 06:40:56.804099 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:56.804103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:56.804323 (+ 220us) server_negotiation.cc:407] Connection header received
0928 06:40:56.804454 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:56.804457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:56.804518 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:56.804623 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:56.805069 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.805860 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.806611 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.806935 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.807042 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:56.809785 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:33761: BlockingRecv error: recv got EOF from 127.8.151.193:33761 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":75,"threads_started":1}
Sep 28 06:40:56 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:56 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:56.982438 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:56.975480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:48363)
0928 06:40:56.976851 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:56.976867 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:56.976970 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:56.977384 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:56.977387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:56.977618 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:56.977743 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:56.977751 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.978542 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.978546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:56.979407 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:56.979420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.979840 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.979843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:56.979929 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:56.980399 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:56.980413 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:56.982298 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1283,"thread_start_us":121,"threads_started":1}
W20260928 06:40:56.982860 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:56.976636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:48363 (local address 127.8.151.254:41531)
0928 06:40:56.977158 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:56.977162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:56.977174 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:40:56.977207 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:56.977210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:56.977254 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:56.977308 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:56.977852 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.978431 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.979512 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:56.979731 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:56.979804 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:56.982506 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:48363: BlockingRecv error: recv got EOF from 127.8.151.194:48363 (error 108)
Metrics: {"server-negotiator.queue_time_us":432}
Sep 28 06:40:57 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:57 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:57.152971 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:57.147376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:40543)
0928 06:40:57.147654 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:57.147669 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:57.147800 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:57.148089 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:57.148091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:57.148287 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:57.148386 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:57.148392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.149349 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.149353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:57.150037 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:57.150047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.150577 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.150580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:57.150643 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:57.151164 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:57.151181 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:57.152849 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":101,"threads_started":1}
W20260928 06:40:57.153290 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:57.147491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:40543 (local address 127.8.151.254:41531)
0928 06:40:57.147642 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:57.147647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:57.147741 (+ 94us) server_negotiation.cc:407] Connection header received
0928 06:40:57.147926 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:57.147928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:57.147976 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:57.148049 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:57.148502 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.149213 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.150153 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.150461 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.150578 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:57.153011 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:40543: BlockingRecv error: recv got EOF from 127.8.151.195:40543 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 28 06:40:57 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:57 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:57.815982 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:57.810496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:33971)
0928 06:40:57.810775 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:57.810790 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:57.810903 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:57.811173 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:57.811176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:57.811373 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:57.811464 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:57.811470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.812210 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.812214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:57.812774 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:57.812781 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.813209 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.813213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:57.813291 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:57.813815 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:57.813826 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:57.815820 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
W20260928 06:40:57.816237 18023 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:57.810552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:33971 (local address 127.8.151.254:41531)
0928 06:40:57.810790 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:57.810794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:57.810849 (+ 55us) server_negotiation.cc:407] Connection header received
0928 06:40:57.810991 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:57.810994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:57.811040 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:57.811119 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:57.811578 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.812111 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.812881 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.813111 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.813195 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:57.815946 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:33971: BlockingRecv error: recv got EOF from 127.8.151.193:33971 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":69,"threads_started":1}
Sep 28 06:40:57 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:57 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:57.988981 18024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:57.983226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:35665)
0928 06:40:57.983546 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:57.983557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:57.983661 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:57.983943 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:57.983946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:57.984155 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:57.984247 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:57.984253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.984991 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.984995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:57.985546 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:57.985553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.986010 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.986014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:57.986105 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:57.986650 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:57.986661 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:57.988834 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":75,"threads_started":1}
W20260928 06:40:57.989384 18023 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:57.983293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:35665 (local address 127.8.151.254:41531)
0928 06:40:57.983437 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:57.983442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:57.983620 (+ 178us) server_negotiation.cc:407] Connection header received
0928 06:40:57.983798 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:57.983801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:57.983850 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:57.983910 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:57.984353 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.984892 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.985647 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:57.985908 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:57.985993 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:57.989246 (+ 3253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:35665: BlockingRecv error: recv got EOF from 127.8.151.194:35665 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 28 06:40:58 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:58 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:58.159672 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:58.153721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:50123)
0928 06:40:58.154028 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:58.154045 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:58.154160 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:58.154406 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:58.154408 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:58.154639 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:58.154761 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:58.154770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.155596 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.155600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:58.156417 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:58.156429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.156855 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.156859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:58.156943 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:58.157562 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:58.157575 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:58.159509 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":95,"threads_started":1}
W20260928 06:40:58.159761 18023 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:58.153809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:50123 (local address 127.8.151.254:41531)
0928 06:40:58.153945 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:58.153950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:58.154102 (+ 152us) server_negotiation.cc:407] Connection header received
0928 06:40:58.154247 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:58.154249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:58.154294 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:58.154357 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:58.154878 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.155480 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.156527 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.156746 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.156815 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:58.159650 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:50123: BlockingRecv error: recv got EOF from 127.8.151.195:50123 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 28 06:40:58 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:58 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:58.822918 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:58.816928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:40271)
0928 06:40:58.817222 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:58.817236 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:58.817330 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:58.817595 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:58.817597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:58.817831 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:58.817957 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:58.817965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.819006 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.819009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:58.819753 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:58.819763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.820265 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.820269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:58.820360 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:58.820895 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:58.820909 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:58.822756 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":80,"threads_started":1}
W20260928 06:40:58.823021 18027 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:58.817003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:40271 (local address 127.8.151.254:41531)
0928 06:40:58.817279 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:58.817283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:58.817299 (+ 16us) server_negotiation.cc:407] Connection header received
0928 06:40:58.817420 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:58.817423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:58.817474 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:58.817574 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:58.818106 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.818887 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.819865 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.820154 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.820258 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:58.822903 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:40271: BlockingRecv error: recv got EOF from 127.8.151.193:40271 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":76,"threads_started":1}
Sep 28 06:40:58 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:58 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:58.995230 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:58.989751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:49367)
0928 06:40:58.990000 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:58.990015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:58.990119 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:58.990352 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:58.990355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:58.990542 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:58.990641 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:58.990647 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.991473 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.991477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:58.992069 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:58.992078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.992494 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.992498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:58.992574 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:58.993173 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:58.993188 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:58.995055 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
W20260928 06:40:58.995324 18027 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:58.989894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:49367 (local address 127.8.151.254:41531)
0928 06:40:58.990043 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:58.990047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:58.990063 (+ 16us) server_negotiation.cc:407] Connection header received
0928 06:40:58.990206 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:58.990209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:58.990257 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:58.990320 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:58.990747 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.991367 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.992183 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:58.992397 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:58.992480 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:58.995224 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:49367: BlockingRecv error: recv got EOF from 127.8.151.194:49367 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 28 06:40:59 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:59 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:59.166340 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:59.160651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:39115)
0928 06:40:59.160936 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:59.160948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:59.161080 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:59.161350 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:59.161353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:59.161528 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:59.161627 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:59.161633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.162564 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.162567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:59.163180 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:59.163193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.163709 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.163712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:59.163808 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:59.164324 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:59.164339 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:59.166225 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":78,"threads_started":1}
W20260928 06:40:59.166497 18027 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:59.160750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:39115 (local address 127.8.151.254:41531)
0928 06:40:59.160883 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:59.160888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:59.161043 (+ 155us) server_negotiation.cc:407] Connection header received
0928 06:40:59.161150 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:59.161153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:59.161206 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:59.161286 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:59.161741 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.162442 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.163298 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.163605 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.163694 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:59.166382 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:39115: BlockingRecv error: recv got EOF from 127.8.151.195:39115 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 28 06:40:59 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:59 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:40:59.829288 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:59.823657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:42293)
0928 06:40:59.824001 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:59.824014 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:59.824131 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:59.824356 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:59.824358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:59.824530 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:59.824631 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:59.824636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.825587 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.825590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:59.826180 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:59.826190 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.826693 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.826695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:59.826752 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:59.827278 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:40:59.827290 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:40:59.829158 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":86,"threads_started":1}
W20260928 06:40:59.829423 18031 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:59.823717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:42293 (local address 127.8.151.254:41531)
0928 06:40:59.824020 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:59.824025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:59.824065 (+ 40us) server_negotiation.cc:407] Connection header received
0928 06:40:59.824208 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:59.824210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:59.824258 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:59.824349 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:59.824727 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.825463 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.826288 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.826604 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.826693 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:40:59.829299 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:42293: BlockingRecv error: recv got EOF from 127.8.151.193:42293 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":65,"threads_started":1}
Sep 28 06:40:59 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:40:59 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:41:00.002298 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:40:59.996110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:52359)
0928 06:40:59.996632 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0928 06:40:59.996648 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:40:59.996761 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:40:59.997254 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:40:59.997258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:40:59.997465 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:40:59.997586 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:59.997595 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.998540 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.998543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:59.999222 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:40:59.999233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.999770 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.999773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:40:59.999840 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:00.000344 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:41:00.000356 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:00.002172 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":84,"threads_started":1}
W20260928 06:41:00.002435 18031 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:40:59.996199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:52359 (local address 127.8.151.254:41531)
0928 06:40:59.996341 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 06:40:59.996346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:40:59.996719 (+ 373us) server_negotiation.cc:407] Connection header received
0928 06:40:59.996845 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:40:59.996848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:40:59.996899 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:40:59.996981 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:40:59.997687 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.998409 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.999330 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:40:59.999645 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:40:59.999740 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:00.002324 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:52359: BlockingRecv error: recv got EOF from 127.8.151.194:52359 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 28 06:41:00 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:41:00 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.195@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:41:00.173448 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:00.167284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.195:53205)
0928 06:41:00.167569 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:00.167583 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:00.167680 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:00.167946 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:00.167948 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:00.168132 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:00.168232 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:00.168237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:00.169391 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:00.169394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:00.169960 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:00.169969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:00.170531 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:00.170535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:00.170603 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:00.171175 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:41:00.171188 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:00.173283 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":82,"threads_started":1}
W20260928 06:41:00.173884 18031 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:41:00.167375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:53205 (local address 127.8.151.254:41531)
0928 06:41:00.167517 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:00.167523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:00.167647 (+ 124us) server_negotiation.cc:407] Connection header received
0928 06:41:00.167752 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:00.167756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:00.167813 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:00.167899 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:00.168362 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:00.169239 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:00.170074 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:00.170416 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:00.170526 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:00.173768 (+ 3242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.195:53205: BlockingRecv error: recv got EOF from 127.8.151.195:53205 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 28 06:41:00 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:41:00 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.193@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:41:00.839207 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:00.830179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.193:42545)
0928 06:41:00.830452 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:00.830466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:00.830558 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:00.830820 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:00.830823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:00.831045 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:00.831198 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:00.831206 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:00.832311 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:00.832313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:00.832884 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:00.832893 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:00.833491 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:00.833495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:00.833568 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:00.836826 (+ 3258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:41:00.836838 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:00.839006 (+ 2168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":82,"threads_started":1}
W20260928 06:41:00.839490 18035 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:41:00.830281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:42545 (local address 127.8.151.254:41531)
0928 06:41:00.830539 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:00.830543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:00.830560 (+ 17us) server_negotiation.cc:407] Connection header received
0928 06:41:00.830623 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:00.830626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:00.830678 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:00.830777 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:00.831332 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:00.832197 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:00.832987 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:00.833346 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:00.833454 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:00.839227 (+ 5773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.193:42545: BlockingRecv error: recv got EOF from 127.8.151.193:42545 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
Sep 28 06:41:01 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
Sep 28 06:41:01 dist-test-slave-gtr8 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.151.194@KRBTEST.COM for oryx/127.8.151.254@KRBTEST.COM, Server not found in Kerberos database
I20260928 06:41:01.009164 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:01.003275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41531 (local address 127.8.151.194:52227)
0928 06:41:01.003563 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:01.003587 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:01.003697 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:01.003915 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:01.003917 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:01.004126 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:01.004233 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:01.004239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.005235 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.005239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:01.005813 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:01.005822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.006434 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.006437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:01.006525 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:01.007026 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:41:01.007038 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:01.009035 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.151.254:41531: Server oryx/127.8.151.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":86,"threads_started":1}
W20260928 06:41:01.009305 18035 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:41:01.003343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:52227 (local address 127.8.151.254:41531)
0928 06:41:01.003493 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:01.003498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:01.003657 (+ 159us) server_negotiation.cc:407] Connection header received
0928 06:41:01.003771 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:01.003773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:01.003816 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:01.003883 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:01.004358 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.005090 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.005921 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.006239 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.006335 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:01.009189 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.151.194:52227: BlockingRecv error: recv got EOF from 127.8.151.194:52227 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:41:01.072818 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 17559
I20260928 06:41:01.077646 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 17712
I20260928 06:41:01.082129 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 17844
I20260928 06:41:01.086936 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 17489
2026-09-28T06:41:01Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17830 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-28T06:41:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:41:01Z Disabled control of system clock
I20260928 06:41:01.117751 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:46019
--webserver_interface=127.8.151.254
--webserver_port=0
--builtin_ntp_servers=127.8.151.212:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:46019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 06:41:01.226200 18040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:01.226454 18040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:01.226486 18040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:01.230500 18040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:41:01.230561 18040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:01.230575 18040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:01.230590 18040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:41:01.230604 18040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:41:01.235325 18040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:46019
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:46019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:01.236374 18040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:01.237753 18040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:41:01.247641 18040 server_base.cc:1034] running on GCE node
W20260928 06:41:01.247539 18045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:01.247534 18046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:01.247589 18048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:01.248410 18040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:01.249560 18040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:01.250749 18040 hybrid_clock.cc:648] HybridClock initialized: now 1790577661250733 us; error 27 us; skew 500 ppm
I20260928 06:41:01.252681 18040 webserver.cc:460] Webserver started at http://127.8.151.254:33117/ using document root <none> and password file <none>
I20260928 06:41:01.253253 18040 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:01.253314 18040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:01.253508 18040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:01.255599 18040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "bdd6500196994fa9b1966c6692bf5991"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "dc1f9d23c95a21560aa541d57f4dac4e"
server_key_iv: "7ef64da6ad2e0908d4439f1ab398191f"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.256168 18040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "bdd6500196994fa9b1966c6692bf5991"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "dc1f9d23c95a21560aa541d57f4dac4e"
server_key_iv: "7ef64da6ad2e0908d4439f1ab398191f"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.259841 18040 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 06:41:01.262084 18054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:01.263363 18040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:41:01.263464 18040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "bdd6500196994fa9b1966c6692bf5991"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "dc1f9d23c95a21560aa541d57f4dac4e"
server_key_iv: "7ef64da6ad2e0908d4439f1ab398191f"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.263566 18040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:01.275938 18040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:01.276639 18040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:01.276808 18040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:01.283947 18040 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:46019
I20260928 06:41:01.283951 18106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:46019 every 8 connection(s)
I20260928 06:41:01.284930 18040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:41:01.288048 18107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:41:01.293466 18107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991: Bootstrap starting.
I20260928 06:41:01.293571 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 18040
I20260928 06:41:01.293695 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:41:01.293994 8799 external_mini_cluster.cc:1428] Setting key f635b709e3700b7c208f6bff55678664
I20260928 06:41:01.296564 18107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991: Neither blocks nor log segments found. Creating new log.
I20260928 06:41:01.297314 18107 log.cc:824] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991: Log is configured to *not* fsync() on all Append() calls
I20260928 06:41:01.299396 18107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991: No bootstrap required, opened a new log
I20260928 06:41:01.303416 18107 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6500196994fa9b1966c6692bf5991" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46019 } }
I20260928 06:41:01.303673 18107 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:41:01.303745 18107 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd6500196994fa9b1966c6692bf5991, State: Initialized, Role: FOLLOWER
I20260928 06:41:01.304194 18107 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6500196994fa9b1966c6692bf5991" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46019 } }
I20260928 06:41:01.304368 18107 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:41:01.304441 18107 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:41:01.304563 18107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:41:01.305816 18107 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6500196994fa9b1966c6692bf5991" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46019 } }
I20260928 06:41:01.306032 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:01.295499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54734 (local address 127.8.151.254:46019)
0928 06:41:01.296404 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:01.296423 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:01.296469 (+ 46us) server_negotiation.cc:407] Connection header received
0928 06:41:01.297361 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:01.297388 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:01.297931 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:01.298321 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:01.298879 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.300457 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.301699 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.302459 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.302621 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:01.303551 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:01.303558 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:01.303576 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:41:01.304095 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:01.304181 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:41:01.304207 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:01.304304 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:01.304778 (+ 474us) server_negotiation.cc:299] Negotiation successful
0928 06:41:01.305050 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":150,"threads_started":1}
I20260928 06:41:01.306182 18107 leader_election.cc:302] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdd6500196994fa9b1966c6692bf5991; no voters:
I20260928 06:41:01.306510 18107 leader_election.cc:288] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:41:01.306619 18112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:41:01.306879 18112 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [term 1 LEADER]: Becoming Leader. State: Replica: bdd6500196994fa9b1966c6692bf5991, State: Running, Role: LEADER
I20260928 06:41:01.307775 18107 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:41:01.308411 18112 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6500196994fa9b1966c6692bf5991" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46019 } }
I20260928 06:41:01.310086 18113 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdd6500196994fa9b1966c6692bf5991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6500196994fa9b1966c6692bf5991" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46019 } } }
I20260928 06:41:01.310321 18113 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [sys.catalog]: This master's current role is: LEADER
I20260928 06:41:01.309674 18114 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdd6500196994fa9b1966c6692bf5991. Latest consensus state: current_term: 1 leader_uuid: "bdd6500196994fa9b1966c6692bf5991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6500196994fa9b1966c6692bf5991" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46019 } } }
I20260928 06:41:01.310668 18114 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991 [sys.catalog]: This master's current role is: LEADER
I20260928 06:41:01.310673 18127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:41:01.314198 18127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:41:01.321513 18127 catalog_manager.cc:1347] Generated new cluster ID: f719833c6d85465b81310c1acf44254d
I20260928 06:41:01.321648 18127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:41:01.339880 18127 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:41:01.341315 18127 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:41:01.346784 18127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdd6500196994fa9b1966c6692bf5991: Generated new TSK 0
I20260928 06:41:01.347697 18127 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:41:01.354060 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:46019
--builtin_ntp_servers=127.8.151.212:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 06:41:01.504695 18131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:01.504951 18131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:01.505002 18131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:01.509137 18131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:01.509192 18131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:01.509263 18131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:41:01.514062 18131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:46019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:01.515117 18131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:01.516522 18131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:01.525910 18137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:01.525964 18139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:01.526094 18131 server_base.cc:1034] running on GCE node
W20260928 06:41:01.525924 18136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:01.526750 18131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:01.527519 18131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:01.528687 18131 hybrid_clock.cc:648] HybridClock initialized: now 1790577661528668 us; error 31 us; skew 500 ppm
I20260928 06:41:01.530460 18131 webserver.cc:460] Webserver started at http://127.8.151.193:41351/ using document root <none> and password file <none>
I20260928 06:41:01.531009 18131 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:01.531088 18131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:01.531327 18131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:01.533344 18131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "243b5d077337479b9e271d684b3c5cd8"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "1fefa7e9fac8f581fdc2c92da7fbc43a"
server_key_iv: "90a2ed0bac234de65c1d974248141311"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.533859 18131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "243b5d077337479b9e271d684b3c5cd8"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "1fefa7e9fac8f581fdc2c92da7fbc43a"
server_key_iv: "90a2ed0bac234de65c1d974248141311"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.537487 18131 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:41:01.540082 18145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:01.541428 18131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:41:01.541543 18131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "243b5d077337479b9e271d684b3c5cd8"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "1fefa7e9fac8f581fdc2c92da7fbc43a"
server_key_iv: "90a2ed0bac234de65c1d974248141311"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.541641 18131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:01.560395 18131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:01.561065 18131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:01.561239 18131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:01.561815 18131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:41:01.562709 18131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:41:01.562754 18131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:01.562804 18131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:41:01.562830 18131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:01.572137 18131 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:37823
I20260928 06:41:01.572192 18258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:37823 every 8 connection(s)
I20260928 06:41:01.573007 18131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:41:01.581861 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 18131
I20260928 06:41:01.581977 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:41:01.582271 8799 external_mini_cluster.cc:1428] Setting key 35c58dc3d0e2dfabd7e8e3078dd1ee10
I20260928 06:41:01.582397 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:01.574796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:33551 (local address 127.8.151.254:46019)
0928 06:41:01.574922 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:01.574926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:01.576231 (+ 1305us) server_negotiation.cc:407] Connection header received
0928 06:41:01.577672 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:01.577676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:01.577736 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:01.577811 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:01.578707 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.579292 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.580412 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.580644 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.580736 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:01.581847 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:01.581850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:01.581852 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:41:01.581865 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:01.581884 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:41:01.581891 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:01.581955 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:01.582186 (+ 231us) server_negotiation.cc:299] Negotiation successful
0928 06:41:01.582251 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 06:41:01.583504 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:01.575316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:46019 (local address 127.8.151.193:33551)
0928 06:41:01.576064 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:01.576111 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:01.577456 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:01.577990 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:01.577997 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:01.578081 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:01.578564 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:01.578576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.579414 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.579417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:01.580296 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:01.580305 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.580791 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.580796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:01.580951 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:01.581548 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:41:01.581550 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:01.581612 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:41:01.581681 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:41:01.582037 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:41:01.582047 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:41:01.582053 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:41:01.582173 (+ 120us) client_negotiation.cc:240] Negotiation successful
0928 06:41:01.582456 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":145,"threads_started":1}
I20260928 06:41:01.584390 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:46019
--builtin_ntp_servers=127.8.151.212:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 06:41:01.584786 18259 heartbeater.cc:347] Connected to a master server at 127.8.151.254:46019
I20260928 06:41:01.585215 18259 heartbeater.cc:464] Registering TS with master...
I20260928 06:41:01.585750 18259 heartbeater.cc:511] Master 127.8.151.254:46019 requested a full tablet report, sending...
I20260928 06:41:01.587447 18071 ts_manager.cc:194] Registered new tserver with Master: 243b5d077337479b9e271d684b3c5cd8 (127.8.151.193:37823)
I20260928 06:41:01.588863 18071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.193:33551
W20260928 06:41:01.694581 18262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:01.694859 18262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:01.694909 18262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:01.699033 18262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:01.699098 18262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:01.699193 18262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:41:01.704028 18262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:46019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:01.705139 18262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:01.706566 18262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:01.717598 18268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:01.717593 18270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:01.717695 18267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:01.718012 18262 server_base.cc:1034] running on GCE node
I20260928 06:41:01.718541 18262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:01.719304 18262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:01.720480 18262 hybrid_clock.cc:648] HybridClock initialized: now 1790577661720461 us; error 30 us; skew 500 ppm
I20260928 06:41:01.722664 18262 webserver.cc:460] Webserver started at http://127.8.151.194:41371/ using document root <none> and password file <none>
I20260928 06:41:01.723366 18262 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:01.723445 18262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:01.723663 18262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:01.726150 18262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "f7bccf4c99bb43c9a940112b53197e0f"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "c2e49703d6d5519532ad27ad400f083d"
server_key_iv: "199c1b97a1298676e721bf4cf09fd4df"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.726774 18262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "f7bccf4c99bb43c9a940112b53197e0f"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "c2e49703d6d5519532ad27ad400f083d"
server_key_iv: "199c1b97a1298676e721bf4cf09fd4df"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.731128 18262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260928 06:41:01.734038 18276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:01.735457 18262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:41:01.735558 18262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "f7bccf4c99bb43c9a940112b53197e0f"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "c2e49703d6d5519532ad27ad400f083d"
server_key_iv: "199c1b97a1298676e721bf4cf09fd4df"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.735677 18262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:01.767316 18262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:01.768045 18262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:01.768229 18262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:01.768855 18262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:41:01.769796 18262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:41:01.769841 18262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:01.769883 18262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:41:01.769912 18262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:01.779372 18262 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:35823
I20260928 06:41:01.779402 18389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:35823 every 8 connection(s)
I20260928 06:41:01.780262 18262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:41:01.783414 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 18262
I20260928 06:41:01.783529 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:41:01.783860 8799 external_mini_cluster.cc:1428] Setting key e8cebd29fcff7bbf18870d876a252217
I20260928 06:41:01.786078 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:46019
--builtin_ntp_servers=127.8.151.212:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 06:41:01.790727 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:01.782045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:40319 (local address 127.8.151.254:46019)
0928 06:41:01.782189 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:01.782193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:01.783593 (+ 1400us) server_negotiation.cc:407] Connection header received
0928 06:41:01.785237 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:01.785241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:01.785289 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:01.785379 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:01.786561 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.787447 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.788907 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.789135 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.789221 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:01.790269 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:01.790273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:01.790275 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:41:01.790287 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:01.790305 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:41:01.790313 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:01.790371 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:01.790514 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 06:41:01.790575 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 06:41:01.791658 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:01.782624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:46019 (local address 127.8.151.194:40319)
0928 06:41:01.783427 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:01.783483 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:01.784996 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:01.785564 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:01.785573 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:01.785690 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:01.786399 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:01.786419 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.787597 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.787600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:01.788797 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:01.788806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.789266 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.789269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:01.789429 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:01.789972 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:41:01.789973 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:01.790029 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:41:01.790096 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:41:01.790382 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:41:01.790391 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:41:01.790396 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 06:41:01.790506 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 06:41:01.790778 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":136,"threads_started":1}
I20260928 06:41:01.792702 18390 heartbeater.cc:347] Connected to a master server at 127.8.151.254:46019
I20260928 06:41:01.792919 18390 heartbeater.cc:464] Registering TS with master...
I20260928 06:41:01.793426 18390 heartbeater.cc:511] Master 127.8.151.254:46019 requested a full tablet report, sending...
I20260928 06:41:01.794355 18071 ts_manager.cc:194] Registered new tserver with Master: f7bccf4c99bb43c9a940112b53197e0f (127.8.151.194:35823)
I20260928 06:41:01.794901 18071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.194:40319
W20260928 06:41:01.913479 18393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:01.913721 18393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:01.913757 18393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:01.917734 18393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:01.917788 18393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:01.917841 18393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:41:01.922552 18393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:46019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:01.923666 18393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:01.924999 18393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:01.934422 18399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:01.934437 18398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:01.934430 18401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:01.934643 18393 server_base.cc:1034] running on GCE node
I20260928 06:41:01.935236 18393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:01.935883 18393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:01.937088 18393 hybrid_clock.cc:648] HybridClock initialized: now 1790577661937058 us; error 42 us; skew 500 ppm
I20260928 06:41:01.938941 18393 webserver.cc:460] Webserver started at http://127.8.151.195:33019/ using document root <none> and password file <none>
I20260928 06:41:01.939585 18393 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:01.939657 18393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:01.939858 18393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:01.941855 18393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "9e425dfc37714c4ab00ef79b130a838a"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "8fbd7d7dbaff25b51a70ad7ad4b56ece"
server_key_iv: "3ef5c78476c1438c95401ac368a143a5"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.942389 18393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "9e425dfc37714c4ab00ef79b130a838a"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "8fbd7d7dbaff25b51a70ad7ad4b56ece"
server_key_iv: "3ef5c78476c1438c95401ac368a143a5"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.946045 18393 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260928 06:41:01.948690 18407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:01.949957 18393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:41:01.950063 18393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "9e425dfc37714c4ab00ef79b130a838a"
format_stamp: "Formatted at 2026-09-28 06:41:01 on dist-test-slave-gtr8"
server_key: "8fbd7d7dbaff25b51a70ad7ad4b56ece"
server_key_iv: "3ef5c78476c1438c95401ac368a143a5"
server_key_version: "encryptionkey@0"
I20260928 06:41:01.950191 18393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:01.964857 18393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:01.965550 18393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:01.965720 18393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:01.966343 18393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:41:01.967340 18393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:41:01.967396 18393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:01.967499 18393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:41:01.967527 18393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:01.976974 18393 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:39541
I20260928 06:41:01.976989 18520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:39541 every 8 connection(s)
I20260928 06:41:01.977886 18393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:41:01.983198 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 18393
I20260928 06:41:01.983305 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:41:01.983572 8799 external_mini_cluster.cc:1428] Setting key a597575790d50f9f305a8750fe9f44e4
I20260928 06:41:01.987480 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:01.979708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:52833 (local address 127.8.151.254:46019)
0928 06:41:01.979836 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:01.979840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:01.981036 (+ 1196us) server_negotiation.cc:407] Connection header received
0928 06:41:01.982384 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:01.982388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:01.982447 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:01.982521 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:01.983472 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.984069 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.985539 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.985846 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.985939 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:01.986994 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:01.986998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:01.987000 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:41:01.987013 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:01.987043 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:41:01.987052 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:01.987120 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:01.987283 (+ 163us) server_negotiation.cc:299] Negotiation successful
0928 06:41:01.987343 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 06:41:01.988399 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:01.980206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:46019 (local address 127.8.151.195:52833)
0928 06:41:01.980884 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:01.980927 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:01.982178 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:01.982705 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:01.982712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:01.982797 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:01.983334 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:01.983353 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.984192 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.984195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:01.985433 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:01.985444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:01.985976 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:01.985978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:01.986116 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:01.986751 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:41:01.986753 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:01.986792 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:41:01.986854 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:41:01.987124 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:41:01.987131 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:41:01.987156 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 06:41:01.987274 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 06:41:01.987491 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":124,"threads_started":1}
I20260928 06:41:01.989519 18521 heartbeater.cc:347] Connected to a master server at 127.8.151.254:46019
I20260928 06:41:01.989766 18521 heartbeater.cc:464] Registering TS with master...
I20260928 06:41:01.990365 18521 heartbeater.cc:511] Master 127.8.151.254:46019 requested a full tablet report, sending...
I20260928 06:41:01.991492 18071 ts_manager.cc:194] Registered new tserver with Master: 9e425dfc37714c4ab00ef79b130a838a (127.8.151.195:39541)
I20260928 06:41:01.992146 18071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.195:52833
I20260928 06:41:01.997587 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 06:41:02.000471 18110 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:41:01.999591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54738 (local address 127.8.151.254:46019)
0928 06:41:01.999720 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:01.999724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:01.999889 (+ 165us) server_negotiation.cc:407] Connection header received
0928 06:41:02.000010 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:02.000012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:02.000050 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:02.000120 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:02.000343 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54738: BlockingRecv error: recv got EOF from 127.0.0.1:54738 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 06:41:02.000622 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 18131
I20260928 06:41:02.006436 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 18262
I20260928 06:41:02.011468 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 18393
I20260928 06:41:02.017717 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 18040
2026-09-28T06:41:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (929 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-28T06:41:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:41:02Z Disabled control of system clock
I20260928 06:41:02.049105 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:42619
--webserver_interface=127.8.151.254
--webserver_port=0
--builtin_ntp_servers=127.8.151.212:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:42619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 06:41:02.168445 18532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:02.168712 18532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:02.168753 18532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:02.172814 18532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:41:02.172883 18532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:02.172899 18532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:02.172915 18532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:41:02.172930 18532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:41:02.177886 18532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:42619
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:42619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:02.179019 18532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:02.180467 18532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:02.188755 18537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:02.188894 18540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:02.189013 18532 server_base.cc:1034] running on GCE node
W20260928 06:41:02.188759 18538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:02.189719 18532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:02.190804 18532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:02.191987 18532 hybrid_clock.cc:648] HybridClock initialized: now 1790577662191968 us; error 29 us; skew 500 ppm
I20260928 06:41:02.193805 18532 webserver.cc:460] Webserver started at http://127.8.151.254:41687/ using document root <none> and password file <none>
I20260928 06:41:02.194358 18532 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:02.194434 18532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:02.194643 18532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:02.196688 18532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "16372b354f3148cb8fa4744d37863f3a"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "7ed3cb8c0f339db18277a86e1949a63a"
server_key_iv: "53399ffe3f8ca462726f783788422237"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.197232 18532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "16372b354f3148cb8fa4744d37863f3a"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "7ed3cb8c0f339db18277a86e1949a63a"
server_key_iv: "53399ffe3f8ca462726f783788422237"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.200826 18532 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260928 06:41:02.203022 18546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:02.204546 18532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260928 06:41:02.204658 18532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "16372b354f3148cb8fa4744d37863f3a"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "7ed3cb8c0f339db18277a86e1949a63a"
server_key_iv: "53399ffe3f8ca462726f783788422237"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.204785 18532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:02.209306 18532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:02.209925 18532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:02.210078 18532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:02.217216 18532 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:42619
I20260928 06:41:02.217232 18598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:42619 every 8 connection(s)
I20260928 06:41:02.218106 18532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:41:02.221120 18599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:41:02.224988 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 18532
I20260928 06:41:02.225104 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:41:02.225394 8799 external_mini_cluster.cc:1428] Setting key 54f9e1a62519b79ba85d824433638c10
I20260928 06:41:02.226986 18599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a: Bootstrap starting.
I20260928 06:41:02.229190 18599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a: Neither blocks nor log segments found. Creating new log.
I20260928 06:41:02.230325 18599 log.cc:824] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a: Log is configured to *not* fsync() on all Append() calls
I20260928 06:41:02.231594 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:02.226872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.8.151.254:42619)
0928 06:41:02.227612 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:02.227630 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:02.227673 (+ 43us) server_negotiation.cc:407] Connection header received
0928 06:41:02.228482 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:02.228506 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:02.228961 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:02.229403 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:02.230047 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:02.230053 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:02.230067 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:41:02.230076 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:02.230134 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:41:02.230153 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:02.230262 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:02.230393 (+ 131us) server_negotiation.cc:299] Negotiation successful
0928 06:41:02.230701 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":126,"threads_started":1}
I20260928 06:41:02.232445 18599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a: No bootstrap required, opened a new log
I20260928 06:41:02.235996 18599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16372b354f3148cb8fa4744d37863f3a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42619 } }
I20260928 06:41:02.236220 18599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:41:02.236270 18599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16372b354f3148cb8fa4744d37863f3a, State: Initialized, Role: FOLLOWER
I20260928 06:41:02.236701 18599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16372b354f3148cb8fa4744d37863f3a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42619 } }
I20260928 06:41:02.236860 18599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:41:02.236927 18599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:41:02.237022 18599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:41:02.237907 18599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16372b354f3148cb8fa4744d37863f3a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42619 } }
I20260928 06:41:02.238233 18599 leader_election.cc:302] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16372b354f3148cb8fa4744d37863f3a; no voters:
I20260928 06:41:02.238518 18599 leader_election.cc:288] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:41:02.238623 18604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:41:02.238842 18604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [term 1 LEADER]: Becoming Leader. State: Replica: 16372b354f3148cb8fa4744d37863f3a, State: Running, Role: LEADER
I20260928 06:41:02.239166 18604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16372b354f3148cb8fa4744d37863f3a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42619 } }
I20260928 06:41:02.239779 18599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:41:02.240794 18606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16372b354f3148cb8fa4744d37863f3a. Latest consensus state: current_term: 1 leader_uuid: "16372b354f3148cb8fa4744d37863f3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16372b354f3148cb8fa4744d37863f3a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42619 } } }
I20260928 06:41:02.240913 18606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [sys.catalog]: This master's current role is: LEADER
I20260928 06:41:02.241232 18605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16372b354f3148cb8fa4744d37863f3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16372b354f3148cb8fa4744d37863f3a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42619 } } }
I20260928 06:41:02.241310 18605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a [sys.catalog]: This master's current role is: LEADER
W20260928 06:41:02.244407 18619 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:41:02.244482 18619 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:41:02.244540 18613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:41:02.245200 18613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:41:02.250036 18613 catalog_manager.cc:1347] Generated new cluster ID: 3aaf2a6e3a07472a8f8753440223b0a9
I20260928 06:41:02.250102 18613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:41:02.262984 18613 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:41:02.263304 18613 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:41:02.278506 18613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16372b354f3148cb8fa4744d37863f3a: Generated new TSK 0
I20260928 06:41:02.279258 18613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:41:02.290547 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42619
--builtin_ntp_servers=127.8.151.212:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 06:41:02.398391 18623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:02.398645 18623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:02.398700 18623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:02.402690 18623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:02.402753 18623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:02.402828 18623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:41:02.407641 18623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:02.408785 18623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:02.410148 18623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:02.419416 18628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:02.419458 18623 server_base.cc:1034] running on GCE node
W20260928 06:41:02.419755 18631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:02.419836 18629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:02.420303 18623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:02.421010 18623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:02.422212 18623 hybrid_clock.cc:648] HybridClock initialized: now 1790577662422185 us; error 43 us; skew 500 ppm
I20260928 06:41:02.424440 18623 webserver.cc:460] Webserver started at http://127.8.151.193:39399/ using document root <none> and password file <none>
I20260928 06:41:02.425124 18623 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:02.425203 18623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:02.425415 18623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:02.428246 18623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "bb377c5e49a04347ba6b1084ddb2c944"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "2d18e1b911111748a8941536dd6d8024"
server_key_iv: "328e025dcb0fd59e9d76137deb8aeec0"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.428946 18623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "bb377c5e49a04347ba6b1084ddb2c944"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "2d18e1b911111748a8941536dd6d8024"
server_key_iv: "328e025dcb0fd59e9d76137deb8aeec0"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.433424 18623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260928 06:41:02.436426 18637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:02.437929 18623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 06:41:02.438050 18623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "bb377c5e49a04347ba6b1084ddb2c944"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "2d18e1b911111748a8941536dd6d8024"
server_key_iv: "328e025dcb0fd59e9d76137deb8aeec0"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.438159 18623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:02.443632 18623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:02.444203 18623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:02.444351 18623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:02.444868 18623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:41:02.445724 18623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:41:02.445767 18623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:02.445802 18623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:41:02.445817 18623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:02.454659 18623 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:43279
I20260928 06:41:02.454681 18750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:43279 every 8 connection(s)
I20260928 06:41:02.455626 18623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:41:02.456672 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 18623
I20260928 06:41:02.456768 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:41:02.457050 8799 external_mini_cluster.cc:1428] Setting key 0732cb933b3b3d6282be3f1cf747aa0e
I20260928 06:41:02.459270 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42619
--builtin_ntp_servers=127.8.151.212:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260928 06:41:02.462191 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:02.457636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:49907 (local address 127.8.151.254:42619)
0928 06:41:02.457781 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:02.457785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:02.458905 (+ 1120us) server_negotiation.cc:407] Connection header received
0928 06:41:02.460292 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:02.460295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:02.460340 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:02.460414 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:02.461703 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:02.461707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:02.461710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:41:02.461724 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:02.461747 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:41:02.461757 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:02.461821 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:02.462004 (+ 183us) server_negotiation.cc:299] Negotiation successful
0928 06:41:02.462060 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 06:41:02.463124 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:02.458058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42619 (local address 127.8.151.193:49907)
0928 06:41:02.458765 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:02.458818 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:02.460091 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:02.460628 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:02.460638 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:02.460760 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:02.461367 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:41:02.461369 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:02.461424 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:41:02.461514 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:41:02.461879 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:41:02.461885 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:41:02.461889 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 06:41:02.462031 (+ 142us) client_negotiation.cc:240] Negotiation successful
0928 06:41:02.462349 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":109,"threads_started":1}
I20260928 06:41:02.464334 18751 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42619
I20260928 06:41:02.464553 18751 heartbeater.cc:464] Registering TS with master...
I20260928 06:41:02.464938 18751 heartbeater.cc:511] Master 127.8.151.254:42619 requested a full tablet report, sending...
I20260928 06:41:02.466308 18556 ts_manager.cc:194] Registered new tserver with Master: bb377c5e49a04347ba6b1084ddb2c944 (127.8.151.193:43279)
W20260928 06:41:02.579386 18754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:02.579656 18754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:02.579715 18754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:02.583662 18754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:02.583715 18754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:02.583791 18754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:41:02.588565 18754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:02.589609 18754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:02.590965 18754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:41:02.601648 18754 server_base.cc:1034] running on GCE node
W20260928 06:41:02.601539 18762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:02.601656 18759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:02.601857 18760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:02.602311 18754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:02.602928 18754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:02.604231 18754 hybrid_clock.cc:648] HybridClock initialized: now 1790577662604191 us; error 57 us; skew 500 ppm
I20260928 06:41:02.606129 18754 webserver.cc:460] Webserver started at http://127.8.151.194:38021/ using document root <none> and password file <none>
I20260928 06:41:02.606684 18754 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:02.606746 18754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:02.606931 18754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:02.609081 18754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "d8e25937baa2474da34964e3f8fab5c4"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "23e619bbdb080b4f6f6c721b7f887b89"
server_key_iv: "908cfb2c32279c8fbcd17cfd87024ffc"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.609603 18754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "d8e25937baa2474da34964e3f8fab5c4"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "23e619bbdb080b4f6f6c721b7f887b89"
server_key_iv: "908cfb2c32279c8fbcd17cfd87024ffc"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.613332 18754 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:41:02.616273 18768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:02.617647 18754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:41:02.617759 18754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "d8e25937baa2474da34964e3f8fab5c4"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "23e619bbdb080b4f6f6c721b7f887b89"
server_key_iv: "908cfb2c32279c8fbcd17cfd87024ffc"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.617861 18754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:02.622335 18754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:02.622932 18754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:02.623091 18754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:02.623654 18754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:41:02.624514 18754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:41:02.624562 18754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:02.624611 18754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:41:02.624636 18754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:02.634025 18754 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:42287
I20260928 06:41:02.634065 18881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:42287 every 8 connection(s)
I20260928 06:41:02.634902 18754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:41:02.636641 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 18754
I20260928 06:41:02.636760 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:41:02.637066 8799 external_mini_cluster.cc:1428] Setting key 09cc3391f12221654546583155a251a3
I20260928 06:41:02.639312 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42619
--builtin_ntp_servers=127.8.151.212:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260928 06:41:02.641882 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:02.637593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:58599 (local address 127.8.151.254:42619)
0928 06:41:02.637745 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:02.637751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:02.638349 (+ 598us) server_negotiation.cc:407] Connection header received
0928 06:41:02.640007 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:02.640010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:02.640051 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:02.640119 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:02.641406 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:02.641410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:02.641413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:41:02.641426 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:02.641451 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:41:02.641462 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:02.641559 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:02.641700 (+ 141us) server_negotiation.cc:299] Negotiation successful
0928 06:41:02.641742 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 06:41:02.643013 18884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:02.637503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42619 (local address 127.8.151.194:58599)
0928 06:41:02.638212 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:02.638252 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:02.639678 (+ 1426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:02.640361 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:02.640369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:02.640472 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:02.641082 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:41:02.641084 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:02.641143 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:41:02.641234 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:41:02.641552 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:41:02.641560 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:41:02.641566 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:41:02.641768 (+ 202us) client_negotiation.cc:240] Negotiation successful
0928 06:41:02.642033 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":123,"threads_started":1}
I20260928 06:41:02.644443 18882 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42619
I20260928 06:41:02.644687 18882 heartbeater.cc:464] Registering TS with master...
I20260928 06:41:02.645162 18882 heartbeater.cc:511] Master 127.8.151.254:42619 requested a full tablet report, sending...
I20260928 06:41:02.646320 18556 ts_manager.cc:194] Registered new tserver with Master: d8e25937baa2474da34964e3f8fab5c4 (127.8.151.194:42287)
W20260928 06:41:02.752219 18885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:02.752490 18885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:02.752542 18885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:02.756618 18885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:02.756685 18885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:02.756745 18885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:41:02.761566 18885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:02.762658 18885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:02.764380 18885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:41:02.774575 18885 server_base.cc:1034] running on GCE node
W20260928 06:41:02.774516 18890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:02.774488 18891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:02.774783 18893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:02.775394 18885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:02.776072 18885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:02.777244 18885 hybrid_clock.cc:648] HybridClock initialized: now 1790577662777210 us; error 52 us; skew 500 ppm
I20260928 06:41:02.779246 18885 webserver.cc:460] Webserver started at http://127.8.151.195:42525/ using document root <none> and password file <none>
I20260928 06:41:02.779816 18885 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:02.779906 18885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:02.780117 18885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:02.782199 18885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "8d82cd53722047f7b1c1dd872288c896"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "5c7a1116b266683e7be01f115d83f1df"
server_key_iv: "c5c241f850311ab80cf9efa97e035b4b"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.782737 18885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "8d82cd53722047f7b1c1dd872288c896"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "5c7a1116b266683e7be01f115d83f1df"
server_key_iv: "c5c241f850311ab80cf9efa97e035b4b"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.786554 18885 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:41:02.789366 18899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:02.790655 18885 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:41:02.790766 18885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "8d82cd53722047f7b1c1dd872288c896"
format_stamp: "Formatted at 2026-09-28 06:41:02 on dist-test-slave-gtr8"
server_key: "5c7a1116b266683e7be01f115d83f1df"
server_key_iv: "c5c241f850311ab80cf9efa97e035b4b"
server_key_version: "encryptionkey@0"
I20260928 06:41:02.790869 18885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:02.795331 18885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:02.795949 18885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:02.796097 18885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:02.796626 18885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:41:02.797515 18885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:41:02.797559 18885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:02.797608 18885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:41:02.797636 18885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:02.806339 18885 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:46645
I20260928 06:41:02.806357 19012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:46645 every 8 connection(s)
I20260928 06:41:02.807330 18885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:41:02.813326 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:02.809132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:45947 (local address 127.8.151.254:42619)
0928 06:41:02.809266 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:02.809270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:02.810456 (+ 1186us) server_negotiation.cc:407] Connection header received
0928 06:41:02.811775 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:02.811778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:02.811824 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:02.811895 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:02.812897 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:02.812900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:02.812903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:41:02.812916 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:02.812936 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:41:02.812944 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:02.813018 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:02.813138 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 06:41:02.813193 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:41:02.814141 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:02.809645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42619 (local address 127.8.151.195:45947)
0928 06:41:02.810311 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:02.810351 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:02.811567 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:02.812051 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:02.812058 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:02.812145 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:02.812643 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:41:02.812644 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:02.812693 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:41:02.812759 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:41:02.813011 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:41:02.813018 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:41:02.813023 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 06:41:02.813176 (+ 153us) client_negotiation.cc:240] Negotiation successful
0928 06:41:02.813408 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":124,"threads_started":1}
I20260928 06:41:02.815264 19013 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42619
I20260928 06:41:02.815466 19013 heartbeater.cc:464] Registering TS with master...
I20260928 06:41:02.815852 19013 heartbeater.cc:511] Master 127.8.151.254:42619 requested a full tablet report, sending...
I20260928 06:41:02.816320 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 18885
I20260928 06:41:02.816416 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:41:02.816684 8799 external_mini_cluster.cc:1428] Setting key 76503b3c984c421451ca353b77a9dbf5
I20260928 06:41:02.817461 18558 ts_manager.cc:194] Registered new tserver with Master: 8d82cd53722047f7b1c1dd872288c896 (127.8.151.195:46645)
I20260928 06:41:02.818521 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 06:41:02.821323 18602 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:41:02.820553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.8.151.254:42619)
0928 06:41:02.820662 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:02.820667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:02.820821 (+ 154us) server_negotiation.cc:407] Connection header received
0928 06:41:02.820926 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:02.820928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:02.820956 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:02.821013 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:02.821209 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43306: BlockingRecv error: recv got EOF from 127.0.0.1:43306 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 06:41:02.821859 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 18623
I20260928 06:41:02.828092 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 18754
I20260928 06:41:02.832813 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 18885
I20260928 06:41:02.838275 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 18532
2026-09-28T06:41:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (818 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:41:02 dist-test-slave-gtr8 krb5kdc[19023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:41:02 dist-test-slave-gtr8 krb5kdc[19023](info): set up 2 sockets
Sep 28 06:41:02 dist-test-slave-gtr8 krb5kdc[19023](info): commencing operation
krb5kdc: starting...
W20260928 06:41:03.824488 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:41:03 dist-test-slave-gtr8 krb5kdc[19023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:41:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:41:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:41:03.950012 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:45797
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:45797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:41:04.084455 19036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:04.084731 19036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:04.084772 19036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:04.088874 19036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:41:04.088945 19036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:04.088963 19036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:04.088979 19036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:41:04.088994 19036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:41:04.094501 19036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:45797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:45797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:04.095674 19036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:04.096710 19036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:04.104948 19041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:04.104966 19044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:04.105187 19036 server_base.cc:1034] running on GCE node
W20260928 06:41:04.104962 19042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:04.105813 19036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:04.106914 19036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:04.108158 19036 hybrid_clock.cc:648] HybridClock initialized: now 1790577664108073 us; error 38 us; skew 500 ppm
Sep 28 06:41:04 dist-test-slave-gtr8 krb5kdc[19023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577664, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:41:04.110812 19036 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:41:04.111814 19036 webserver.cc:460] Webserver started at http://127.8.151.254:44061/ using document root <none> and password file <none>
I20260928 06:41:04.112350 19036 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:04.112409 19036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:04.112604 19036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:04.114629 19036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "6385de874ef34c958020e1a7e6163b71"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "a576c8a2aadccd24d655fe76264046a8"
server_key_iv: "0bef14c0d4d255730fe7aac86d3ad00d"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.115216 19036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "6385de874ef34c958020e1a7e6163b71"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "a576c8a2aadccd24d655fe76264046a8"
server_key_iv: "0bef14c0d4d255730fe7aac86d3ad00d"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.118881 19036 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:41:04.121266 19051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:04.122515 19036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:41:04.122618 19036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "6385de874ef34c958020e1a7e6163b71"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "a576c8a2aadccd24d655fe76264046a8"
server_key_iv: "0bef14c0d4d255730fe7aac86d3ad00d"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.122728 19036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:04.138401 19036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:04.141466 19036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:04.141639 19036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:04.148938 19036 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:45797
I20260928 06:41:04.149211 19103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:45797 every 8 connection(s)
I20260928 06:41:04.149981 19036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:41:04.152804 19104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:41:04.156738 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 19036
I20260928 06:41:04.156864 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:41:04.157189 8799 external_mini_cluster.cc:1428] Setting key 8f5ce28880f6e70efc7fd45c0c6a6c82
I20260928 06:41:04.159017 19104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71: Bootstrap starting.
I20260928 06:41:04.161382 19104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71: Neither blocks nor log segments found. Creating new log.
I20260928 06:41:04.162186 19104 log.cc:824] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71: Log is configured to *not* fsync() on all Append() calls
I20260928 06:41:04.164124 19104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71: No bootstrap required, opened a new log
Sep 28 06:41:04 dist-test-slave-gtr8 krb5kdc[19023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:41:04.168125 19104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6385de874ef34c958020e1a7e6163b71" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45797 } }
I20260928 06:41:04.168345 19104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:41:04.168396 19104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6385de874ef34c958020e1a7e6163b71, State: Initialized, Role: FOLLOWER
I20260928 06:41:04.168826 19104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6385de874ef34c958020e1a7e6163b71" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45797 } }
I20260928 06:41:04.168974 19104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:41:04.169040 19104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:41:04.169137 19104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:41:04.170071 19104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6385de874ef34c958020e1a7e6163b71" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45797 } }
I20260928 06:41:04.170405 19104 leader_election.cc:302] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6385de874ef34c958020e1a7e6163b71; no voters:
I20260928 06:41:04.170706 19104 leader_election.cc:288] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:41:04.170883 19109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:41:04.171106 19109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [term 1 LEADER]: Becoming Leader. State: Replica: 6385de874ef34c958020e1a7e6163b71, State: Running, Role: LEADER
I20260928 06:41:04.171479 19109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6385de874ef34c958020e1a7e6163b71" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45797 } }
I20260928 06:41:04.172019 19104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:41:04.172763 19109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6385de874ef34c958020e1a7e6163b71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6385de874ef34c958020e1a7e6163b71" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45797 } } }
I20260928 06:41:04.172964 19109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [sys.catalog]: This master's current role is: LEADER
I20260928 06:41:04.173214 19109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6385de874ef34c958020e1a7e6163b71. Latest consensus state: current_term: 1 leader_uuid: "6385de874ef34c958020e1a7e6163b71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6385de874ef34c958020e1a7e6163b71" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45797 } } }
I20260928 06:41:04.173350 19109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71 [sys.catalog]: This master's current role is: LEADER
I20260928 06:41:04.173676 19118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:41:04.174719 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.158661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50216 (local address 127.8.151.254:45797)
0928 06:41:04.159232 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:04.159252 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:04.159305 (+ 53us) server_negotiation.cc:407] Connection header received
0928 06:41:04.160128 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:04.160152 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:04.160596 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:04.160954 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:04.161819 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.163447 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.164576 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.165319 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.165484 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.168505 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:04.168511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:04.168530 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:41:04.168564 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:04.170665 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:04.171238 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:04.171244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:04.171251 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:04.171410 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:04.171727 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:04.171730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:04.171732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:04.172256 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:04.173168 (+ 912us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:04.173342 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 06:41:04.173662 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":156,"threads_started":1}
I20260928 06:41:04.178359 19118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:41:04.183705 19118 catalog_manager.cc:1347] Generated new cluster ID: ef0a6a0a81af48c28c39c80cc66f64bd
I20260928 06:41:04.183779 19118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:41:04.193509 19118 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:41:04.194516 19118 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:41:04.207370 19118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71: Generated new TSK 0
I20260928 06:41:04.208112 19118 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:41:04.238119 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45797
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:41:04.347263 19130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:04.347520 19130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:04.347587 19130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:04.351543 19130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:04.351605 19130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:04.351698 19130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:41:04.356952 19130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:04.357990 19130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:04.359018 19130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:04.368887 19138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:04.368968 19136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:04.369023 19135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:04.369493 19130 server_base.cc:1034] running on GCE node
I20260928 06:41:04.369879 19130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:04.370483 19130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:04.371671 19130 hybrid_clock.cc:648] HybridClock initialized: now 1790577664371635 us; error 35 us; skew 500 ppm
Sep 28 06:41:04 dist-test-slave-gtr8 krb5kdc[19023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577664, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:41:04.374500 19130 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:41:04.375427 19130 webserver.cc:460] Webserver started at http://127.8.151.193:36449/ using document root <none> and password file <none>
I20260928 06:41:04.375982 19130 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:04.376046 19130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:04.376258 19130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:04.378261 19130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "baa8e36784a04ce29359e070394f97e7"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "a5efefc178578eee773fb013034676c4"
server_key_iv: "ecf586368643460c5035554eb127b533"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.378791 19130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "baa8e36784a04ce29359e070394f97e7"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "a5efefc178578eee773fb013034676c4"
server_key_iv: "ecf586368643460c5035554eb127b533"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.382548 19130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:41:04.385178 19145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:04.386473 19130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:41:04.386581 19130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "baa8e36784a04ce29359e070394f97e7"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "a5efefc178578eee773fb013034676c4"
server_key_iv: "ecf586368643460c5035554eb127b533"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.386674 19130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:04.403266 19130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:04.406157 19130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:04.406327 19130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:04.406885 19130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:41:04.407702 19130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:41:04.407749 19130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:04.407799 19130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:41:04.407827 19130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:04.417143 19130 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:37789
I20260928 06:41:04.417173 19258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:37789 every 8 connection(s)
I20260928 06:41:04.418005 19130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:41:04.424038 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 19130
I20260928 06:41:04.424154 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:41:04.424443 8799 external_mini_cluster.cc:1428] Setting key 8fc5c5eb527da4c45d159a39296c5cee
Sep 28 06:41:04 dist-test-slave-gtr8 krb5kdc[19023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577664, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:41:04.432541 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.419892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:49013 (local address 127.8.151.254:45797)
0928 06:41:04.420019 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:04.420023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:04.421014 (+ 991us) server_negotiation.cc:407] Connection header received
0928 06:41:04.422368 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:04.422372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:04.422434 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:04.422519 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:04.423782 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.424396 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.425930 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.426262 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.426359 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.429032 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:04.429036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:04.429039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:41:04.429073 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:04.430484 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:04.430981 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:04.430985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:04.430987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:04.431077 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:04.431425 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:04.431429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:04.431430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:04.431620 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:04.431716 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:04.432225 (+ 509us) server_negotiation.cc:299] Negotiation successful
0928 06:41:04.432358 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 06:41:04.433466 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.420315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45797 (local address 127.8.151.193:49013)
0928 06:41:04.420843 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:04.420892 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:04.422153 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:04.422713 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:04.422719 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:04.423110 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:04.423636 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.423651 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.424583 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.424586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.425767 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.425779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.426424 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.426429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.426627 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.427257 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:41:04.427280 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:04.428861 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:41:04.430599 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:41:04.430605 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:41:04.430621 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:41:04.430881 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:41:04.431182 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:41:04.431186 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:41:04.431188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:41:04.431333 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:41:04.431724 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:41:04.431730 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:41:04.431976 (+ 246us) client_negotiation.cc:769] Sending connection context
0928 06:41:04.432146 (+ 170us) client_negotiation.cc:240] Negotiation successful
0928 06:41:04.432470 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:41:04.434692 19259 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45797
I20260928 06:41:04.434928 19259 heartbeater.cc:464] Registering TS with master...
I20260928 06:41:04.435478 19259 heartbeater.cc:511] Master 127.8.151.254:45797 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:41:04.437199 19068 ts_manager.cc:194] Registered new tserver with Master: baa8e36784a04ce29359e070394f97e7 (127.8.151.193:37789)
I20260928 06:41:04.438315 19068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:49013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:41:04.454597 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45797
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:41:04.564244 19264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:04.564507 19264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:04.564548 19264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:04.568570 19264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:04.568627 19264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:04.568698 19264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:41:04.573921 19264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:04.574980 19264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:04.575989 19264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:04.584931 19270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:04.584950 19269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:04.585177 19264 server_base.cc:1034] running on GCE node
W20260928 06:41:04.584944 19272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:04.585686 19264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:04.586256 19264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:04.587415 19264 hybrid_clock.cc:648] HybridClock initialized: now 1790577664587392 us; error 35 us; skew 500 ppm
Sep 28 06:41:04 dist-test-slave-gtr8 krb5kdc[19023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577664, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:41:04.590138 19264 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:41:04.591205 19264 webserver.cc:460] Webserver started at http://127.8.151.194:42361/ using document root <none> and password file <none>
I20260928 06:41:04.591869 19264 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:04.591946 19264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:04.592173 19264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:04.594805 19264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "dadae0a7023e4a6486f5c6ccf41622f4"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "5f89677fef741a6940f90dc74a066c6c"
server_key_iv: "231bd8c258945fd461712d463a975f10"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.595484 19264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "dadae0a7023e4a6486f5c6ccf41622f4"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "5f89677fef741a6940f90dc74a066c6c"
server_key_iv: "231bd8c258945fd461712d463a975f10"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.600243 19264 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260928 06:41:04.603240 19279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:04.604671 19264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:41:04.604779 19264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "dadae0a7023e4a6486f5c6ccf41622f4"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "5f89677fef741a6940f90dc74a066c6c"
server_key_iv: "231bd8c258945fd461712d463a975f10"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.604890 19264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:04.632308 19264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:04.636118 19264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:04.636312 19264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:04.637020 19264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:41:04.638026 19264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:41:04.638087 19264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:04.638139 19264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:41:04.638167 19264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:04.648226 19264 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:35347
I20260928 06:41:04.648268 19392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:35347 every 8 connection(s)
I20260928 06:41:04.649236 19264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:41:04.651058 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 19264
I20260928 06:41:04.651186 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:41:04.651484 8799 external_mini_cluster.cc:1428] Setting key 75a34d55c55e30436ad327ed602c4646
Sep 28 06:41:04 dist-test-slave-gtr8 krb5kdc[19023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577664, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
I20260928 06:41:04.664283 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.651045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:51065 (local address 127.8.151.254:45797)
0928 06:41:04.651308 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:04.651313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:04.652210 (+ 897us) server_negotiation.cc:407] Connection header received
0928 06:41:04.653747 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:04.653750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:04.653799 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:04.653878 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:04.655216 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.655735 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.657253 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.657552 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.657652 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.660512 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:04.660516 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:04.660519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:41:04.660546 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:04.661885 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:04.662492 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:04.662495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:04.662496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:04.662598 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:04.663018 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:04.663022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:04.663024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:04.663279 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:04.663371 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:04.663980 (+ 609us) server_negotiation.cc:299] Negotiation successful
0928 06:41:04.664107 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 06:41:04.665169 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.651506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45797 (local address 127.8.151.194:51065)
0928 06:41:04.652056 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:04.652113 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:04.653528 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:04.654050 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:04.654060 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:04.654512 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:04.655068 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.655081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.655857 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.655860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.657108 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.657118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.657716 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.657720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.657913 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.658540 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:41:04.658561 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:04.660334 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:41:04.661986 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:41:04.661992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:41:04.662006 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:41:04.662351 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:41:04.662683 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:41:04.662686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:41:04.662687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:41:04.662885 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:41:04.663381 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:41:04.663387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:41:04.663721 (+ 334us) client_negotiation.cc:769] Sending connection context
0928 06:41:04.663902 (+ 181us) client_negotiation.cc:240] Negotiation successful
0928 06:41:04.664206 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":142,"threads_started":1}
I20260928 06:41:04.666371 19393 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45797
I20260928 06:41:04.666613 19393 heartbeater.cc:464] Registering TS with master...
I20260928 06:41:04.667162 19393 heartbeater.cc:511] Master 127.8.151.254:45797 requested a full tablet report, sending...
I20260928 06:41:04.668298 19068 ts_manager.cc:194] Registered new tserver with Master: dadae0a7023e4a6486f5c6ccf41622f4 (127.8.151.194:35347)
I20260928 06:41:04.669103 19068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:51065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:41:04.680143 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45797
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:41:04.786490 19398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:04.786742 19398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:04.786784 19398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:04.790861 19398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:04.790932 19398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:04.791006 19398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:41:04.796370 19398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:04.797399 19398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:04.798383 19398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:04.807687 19403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:04.807735 19404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:04.807688 19406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:04.807927 19398 server_base.cc:1034] running on GCE node
I20260928 06:41:04.808468 19398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:04.809070 19398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:04.810261 19398 hybrid_clock.cc:648] HybridClock initialized: now 1790577664810235 us; error 35 us; skew 500 ppm
Sep 28 06:41:04 dist-test-slave-gtr8 krb5kdc[19023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577664, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:41:04.812978 19398 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:41:04.813858 19398 webserver.cc:460] Webserver started at http://127.8.151.195:34341/ using document root <none> and password file <none>
I20260928 06:41:04.814379 19398 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:04.814445 19398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:04.814638 19398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:41:04.816725 19398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "e589246d2cab40348707ed497b91a52f"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "f41787a3a18353273b6dcb91a3df17ba"
server_key_iv: "51a04ff609c065578dff839d158a3cca"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.817248 19398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "e589246d2cab40348707ed497b91a52f"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "f41787a3a18353273b6dcb91a3df17ba"
server_key_iv: "51a04ff609c065578dff839d158a3cca"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.820986 19398 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:41:04.823357 19413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:04.824613 19398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:41:04.824729 19398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "e589246d2cab40348707ed497b91a52f"
format_stamp: "Formatted at 2026-09-28 06:41:04 on dist-test-slave-gtr8"
server_key: "f41787a3a18353273b6dcb91a3df17ba"
server_key_iv: "51a04ff609c065578dff839d158a3cca"
server_key_version: "encryptionkey@0"
I20260928 06:41:04.824826 19398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:04.838465 19398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:04.841373 19398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:04.841537 19398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:04.842101 19398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:41:04.842898 19398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:41:04.842942 19398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:04.842993 19398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:41:04.843019 19398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:04.852144 19398 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:45149
I20260928 06:41:04.852167 19526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:45149 every 8 connection(s)
I20260928 06:41:04.852977 19398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:41:04.855722 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 19398
I20260928 06:41:04.855855 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:41:04.856107 8799 external_mini_cluster.cc:1428] Setting key de3dad898ba9790d1147e1bb89f53d90
Sep 28 06:41:04 dist-test-slave-gtr8 krb5kdc[19023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577664, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:41:04.868005 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.854699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:41031 (local address 127.8.151.254:45797)
0928 06:41:04.854845 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:04.854850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:04.855781 (+ 931us) server_negotiation.cc:407] Connection header received
0928 06:41:04.857538 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:04.857541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:04.857591 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:04.857672 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:04.859221 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.859746 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.861430 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.861659 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.861754 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.864516 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:04.864519 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:04.864521 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:41:04.864545 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:04.865804 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:04.866377 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:04.866380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:04.866381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:04.866458 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:04.866842 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:04.866846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:04.866847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:04.867034 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:04.867124 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:04.867770 (+ 646us) server_negotiation.cc:299] Negotiation successful
0928 06:41:04.867873 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:41:04.869084 19529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.855044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45797 (local address 127.8.151.195:41031)
0928 06:41:04.855612 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:04.855679 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:04.857318 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:04.857873 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:04.857882 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:04.858353 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:04.859023 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.859042 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.859872 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.859876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.861295 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.861306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.861810 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.861813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.862013 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.862607 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:41:04.862638 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:04.864338 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:41:04.865917 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:41:04.865923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:41:04.865939 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:41:04.866274 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:41:04.866537 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:41:04.866540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:41:04.866542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:41:04.866724 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:41:04.867133 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:41:04.867161 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:41:04.867482 (+ 321us) client_negotiation.cc:769] Sending connection context
0928 06:41:04.867699 (+ 217us) client_negotiation.cc:240] Negotiation successful
0928 06:41:04.868040 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":123,"threads_started":1}
I20260928 06:41:04.870301 19527 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45797
I20260928 06:41:04.870528 19527 heartbeater.cc:464] Registering TS with master...
I20260928 06:41:04.870992 19527 heartbeater.cc:511] Master 127.8.151.254:45797 requested a full tablet report, sending...
I20260928 06:41:04.872014 19068 ts_manager.cc:194] Registered new tserver with Master: e589246d2cab40348707ed497b91a52f (127.8.151.195:45149)
I20260928 06:41:04.872622 19068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:41031
I20260928 06:41:04.881448 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:41:04.891356 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.883361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50232 (local address 127.8.151.254:45797)
0928 06:41:04.883501 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:04.883506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:04.883759 (+ 253us) server_negotiation.cc:407] Connection header received
0928 06:41:04.883862 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:04.883864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:04.883908 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:04.883975 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:04.884813 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.885384 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.886446 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.886658 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.886735 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.888273 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:04.888276 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:04.888278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:41:04.888301 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:04.889560 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:04.890050 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:04.890054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:04.890056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:04.890181 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:04.890449 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:04.890451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:04.890452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:04.890661 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:04.890756 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:04.891017 (+ 261us) server_negotiation.cc:299] Negotiation successful
0928 06:41:04.891197 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 06:41:04.894402 19068 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:41:04.897455 19068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:41:04.912783 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.906222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:35347 (local address 127.0.0.1:47444)
0928 06:41:04.907380 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:04.907433 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:04.907570 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:04.908110 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:04.908117 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:04.908136 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:41:04.908281 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.908287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.909667 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.909670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.911104 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.911115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.912224 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.912227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.912336 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.912390 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 06:41:04.912470 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 06:41:04.912582 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260928 06:41:04.913290 19541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.906529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:45149 (local address 127.0.0.1:58840)
0928 06:41:04.907951 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:04.907964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:04.908052 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:04.908935 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:04.908938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:04.908952 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:41:04.909028 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.909034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.910296 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.910299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.911427 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.911436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.912664 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.912668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.912768 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.912790 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 06:41:04.912854 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 06:41:04.913160 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1360,"thread_start_us":100,"threads_started":1}
I20260928 06:41:04.913463 19539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.906354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47444 (local address 127.8.151.194:35347)
0928 06:41:04.906701 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:04.906712 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:04.907525 (+ 813us) server_negotiation.cc:407] Connection header received
0928 06:41:04.907727 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:04.907734 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:04.907952 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:04.908114 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:41:04.908398 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.909461 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.911289 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.912087 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.913060 (+ 973us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.913196 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:04.913244 (+ 48us) server_negotiation.cc:299] Negotiation successful
0928 06:41:04.913331 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":152,"threads_started":1}
I20260928 06:41:04.914007 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.907566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.8.151.195:45149)
0928 06:41:04.908386 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:04.908400 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:04.908421 (+ 21us) server_negotiation.cc:407] Connection header received
0928 06:41:04.908488 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:04.908493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:04.908657 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:04.908796 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:41:04.909140 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.910161 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.911547 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.912360 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.912461 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.912608 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:04.913769 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0928 06:41:04.913871 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":105,"threads_started":1}
I20260928 06:41:04.915910 19538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.906946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:37789 (local address 127.0.0.1:43170)
0928 06:41:04.908005 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:04.908017 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:04.908100 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:04.908768 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:04.908771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:04.908788 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:41:04.908883 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.908889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.911254 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.911258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.912365 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:04.912374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.915499 (+ 3125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.915503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:04.915602 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.915625 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:41:04.915701 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 06:41:04.915771 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"thread_start_us":85,"threads_started":1}
I20260928 06:41:04.916375 19542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:04.907249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.8.151.193:37789)
0928 06:41:04.907863 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:04.907878 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:04.908187 (+ 309us) server_negotiation.cc:407] Connection header received
0928 06:41:04.908275 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:04.908280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:04.908483 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:04.908611 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:41:04.909004 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.911084 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.912487 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:04.914243 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:04.915300 (+ 1057us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:04.915436 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:04.916145 (+ 709us) server_negotiation.cc:299] Negotiation successful
0928 06:41:04.916243 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":116,"threads_started":1}
I20260928 06:41:04.916738 19327 tablet_service.cc:1467] Processing CreateTablet for tablet a63bf9045a4b4534b4703eb99d76c823 (DEFAULT_TABLE table=test-table [id=3195070bab954cc79609795ccac4943b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:41:04.916791 19461 tablet_service.cc:1467] Processing CreateTablet for tablet a63bf9045a4b4534b4703eb99d76c823 (DEFAULT_TABLE table=test-table [id=3195070bab954cc79609795ccac4943b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:41:04.917747 19327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a63bf9045a4b4534b4703eb99d76c823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:41:04.917785 19461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a63bf9045a4b4534b4703eb99d76c823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:41:04.921954 19193 tablet_service.cc:1467] Processing CreateTablet for tablet a63bf9045a4b4534b4703eb99d76c823 (DEFAULT_TABLE table=test-table [id=3195070bab954cc79609795ccac4943b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:41:04.922989 19193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a63bf9045a4b4534b4703eb99d76c823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:41:04.927695 19545 tablet_bootstrap.cc:492] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: Bootstrap starting.
I20260928 06:41:04.929625 19546 tablet_bootstrap.cc:492] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7: Bootstrap starting.
I20260928 06:41:04.930058 19545 tablet_bootstrap.cc:654] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: Neither blocks nor log segments found. Creating new log.
I20260928 06:41:04.931372 19545 log.cc:824] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: Log is configured to *not* fsync() on all Append() calls
I20260928 06:41:04.931759 19546 tablet_bootstrap.cc:654] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7: Neither blocks nor log segments found. Creating new log.
I20260928 06:41:04.932893 19546 log.cc:824] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7: Log is configured to *not* fsync() on all Append() calls
I20260928 06:41:04.933326 19544 tablet_bootstrap.cc:492] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: Bootstrap starting.
I20260928 06:41:04.933533 19545 tablet_bootstrap.cc:492] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: No bootstrap required, opened a new log
I20260928 06:41:04.933727 19545 ts_tablet_manager.cc:1397] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:41:04.935838 19544 tablet_bootstrap.cc:654] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: Neither blocks nor log segments found. Creating new log.
I20260928 06:41:04.936636 19544 log.cc:824] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: Log is configured to *not* fsync() on all Append() calls
I20260928 06:41:04.937554 19545 raft_consensus.cc:348] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 } } peers { permanent_uuid: "baa8e36784a04ce29359e070394f97e7" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37789 } } peers { permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 } }
I20260928 06:41:04.937842 19545 raft_consensus.cc:374] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:41:04.937923 19545 raft_consensus.cc:729] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e589246d2cab40348707ed497b91a52f, State: Initialized, Role: FOLLOWER
I20260928 06:41:04.938400 19545 consensus_queue.cc:260] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 } } peers { permanent_uuid: "baa8e36784a04ce29359e070394f97e7" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37789 } } peers { permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 } }
I20260928 06:41:04.938632 19544 tablet_bootstrap.cc:492] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: No bootstrap required, opened a new log
I20260928 06:41:04.938833 19544 ts_tablet_manager.cc:1397] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 06:41:04.939116 19527 heartbeater.cc:503] Master 127.8.151.254:45797 was elected leader, sending a full tablet report...
I20260928 06:41:04.939566 19545 ts_tablet_manager.cc:1428] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260928 06:41:04.942483 19544 raft_consensus.cc:348] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 } } peers { permanent_uuid: "baa8e36784a04ce29359e070394f97e7" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37789 } } peers { permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 } }
I20260928 06:41:04.942672 19544 raft_consensus.cc:374] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:41:04.942703 19544 raft_consensus.cc:729] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dadae0a7023e4a6486f5c6ccf41622f4, State: Initialized, Role: FOLLOWER
I20260928 06:41:04.943099 19544 consensus_queue.cc:260] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 } } peers { permanent_uuid: "baa8e36784a04ce29359e070394f97e7" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37789 } } peers { permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 } }
I20260928 06:41:04.943820 19393 heartbeater.cc:503] Master 127.8.151.254:45797 was elected leader, sending a full tablet report...
I20260928 06:41:04.944111 19544 ts_tablet_manager.cc:1428] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:41:04.945559 19546 tablet_bootstrap.cc:492] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7: No bootstrap required, opened a new log
I20260928 06:41:04.945744 19546 ts_tablet_manager.cc:1397] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260928 06:41:04.948846 19546 raft_consensus.cc:348] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 } } peers { permanent_uuid: "baa8e36784a04ce29359e070394f97e7" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37789 } } peers { permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 } }
I20260928 06:41:04.949044 19546 raft_consensus.cc:374] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:41:04.949097 19546 raft_consensus.cc:729] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa8e36784a04ce29359e070394f97e7, State: Initialized, Role: FOLLOWER
I20260928 06:41:04.949513 19546 consensus_queue.cc:260] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 } } peers { permanent_uuid: "baa8e36784a04ce29359e070394f97e7" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37789 } } peers { permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 } }
I20260928 06:41:04.950101 19259 heartbeater.cc:503] Master 127.8.151.254:45797 was elected leader, sending a full tablet report...
I20260928 06:41:04.950346 19546 ts_tablet_manager.cc:1428] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:41:05.049804 19552 raft_consensus.cc:482] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:41:05.050016 19552 raft_consensus.cc:504] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 } } peers { permanent_uuid: "baa8e36784a04ce29359e070394f97e7" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37789 } } peers { permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 } }
I20260928 06:41:05.051270 19552 leader_election.cc:288] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dadae0a7023e4a6486f5c6ccf41622f4 (127.8.151.194:35347), e589246d2cab40348707ed497b91a52f (127.8.151.195:45149)
I20260928 06:41:05.055707 19539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:05.051969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:42221 (local address 127.8.151.194:35347)
0928 06:41:05.052081 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:05.052085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:05.052099 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:41:05.052155 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:05.052159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:05.052209 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:05.052289 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:41:05.052583 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:05.053371 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:05.054686 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:05.055077 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:05.055224 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:05.055277 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:05.055481 (+ 204us) server_negotiation.cc:299] Negotiation successful
0928 06:41:05.055565 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:41:05.055711 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:05.051555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:45149 (local address 127.8.151.193:53349)
0928 06:41:05.051912 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:05.051924 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:05.052028 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:05.052304 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:05.052307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:05.052326 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:41:05.052475 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:05.052481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:05.053228 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:05.053231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:05.054414 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:05.054426 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:05.055347 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:05.055351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:05.055443 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:05.055462 (+ 19us) client_negotiation.cc:769] Sending connection context
0928 06:41:05.055528 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 06:41:05.055578 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":1,"spinlock_wait_cycles":20608,"thread_start_us":141,"threads_started":1}
I20260928 06:41:05.055711 19554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:05.051539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:35347 (local address 127.8.151.193:42221)
0928 06:41:05.051886 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:05.051901 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:05.052028 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:05.052391 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:05.052393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:05.052405 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:41:05.052490 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:05.052494 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:05.053484 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:05.053487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:05.054572 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:05.054579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:05.055224 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:05.055228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:05.055318 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:05.055342 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 06:41:05.055446 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 06:41:05.055519 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"spinlock_wait_cycles":14080,"thread_start_us":135,"threads_started":1}
I20260928 06:41:05.056192 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:05.051668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:53349 (local address 127.8.151.195:45149)
0928 06:41:05.051813 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:05.051818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:05.052114 (+ 296us) server_negotiation.cc:407] Connection header received
0928 06:41:05.052168 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:05.052170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:05.052209 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:05.052269 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:41:05.052619 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:05.053113 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:05.054555 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:05.055242 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:05.055355 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:05.055405 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:05.055770 (+ 365us) server_negotiation.cc:299] Negotiation successful
0928 06:41:05.056080 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:41:05.056694 19481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63bf9045a4b4534b4703eb99d76c823" candidate_uuid: "baa8e36784a04ce29359e070394f97e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e589246d2cab40348707ed497b91a52f" is_pre_election: true
I20260928 06:41:05.056708 19347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63bf9045a4b4534b4703eb99d76c823" candidate_uuid: "baa8e36784a04ce29359e070394f97e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" is_pre_election: true
I20260928 06:41:05.056991 19481 raft_consensus.cc:2454] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baa8e36784a04ce29359e070394f97e7 in term 0.
I20260928 06:41:05.056993 19347 raft_consensus.cc:2454] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baa8e36784a04ce29359e070394f97e7 in term 0.
I20260928 06:41:05.057430 19147 leader_election.cc:302] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: baa8e36784a04ce29359e070394f97e7, e589246d2cab40348707ed497b91a52f; no voters:
I20260928 06:41:05.057713 19552 raft_consensus.cc:2781] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:41:05.057793 19552 raft_consensus.cc:482] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:41:05.057833 19552 raft_consensus.cc:3037] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:41:05.058812 19552 raft_consensus.cc:504] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 } } peers { permanent_uuid: "baa8e36784a04ce29359e070394f97e7" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37789 } } peers { permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 } }
I20260928 06:41:05.059122 19552 leader_election.cc:288] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [CANDIDATE]: Term 1 election: Requested vote from peers dadae0a7023e4a6486f5c6ccf41622f4 (127.8.151.194:35347), e589246d2cab40348707ed497b91a52f (127.8.151.195:45149)
I20260928 06:41:05.059468 19481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63bf9045a4b4534b4703eb99d76c823" candidate_uuid: "baa8e36784a04ce29359e070394f97e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e589246d2cab40348707ed497b91a52f"
I20260928 06:41:05.059490 19347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63bf9045a4b4534b4703eb99d76c823" candidate_uuid: "baa8e36784a04ce29359e070394f97e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dadae0a7023e4a6486f5c6ccf41622f4"
I20260928 06:41:05.059600 19481 raft_consensus.cc:3037] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:41:05.059633 19347 raft_consensus.cc:3037] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:41:05.060561 19481 raft_consensus.cc:2454] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa8e36784a04ce29359e070394f97e7 in term 1.
I20260928 06:41:05.060832 19147 leader_election.cc:302] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: baa8e36784a04ce29359e070394f97e7, e589246d2cab40348707ed497b91a52f; no voters:
I20260928 06:41:05.060917 19347 raft_consensus.cc:2454] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa8e36784a04ce29359e070394f97e7 in term 1.
I20260928 06:41:05.060990 19552 raft_consensus.cc:2781] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:41:05.061213 19552 raft_consensus.cc:686] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [term 1 LEADER]: Becoming Leader. State: Replica: baa8e36784a04ce29359e070394f97e7, State: Running, Role: LEADER
I20260928 06:41:05.061604 19552 consensus_queue.cc:237] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 } } peers { permanent_uuid: "baa8e36784a04ce29359e070394f97e7" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37789 } } peers { permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 } }
I20260928 06:41:05.064620 19068 catalog_manager.cc:5495] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to baa8e36784a04ce29359e070394f97e7 (127.8.151.193). New cstate: current_term: 1 leader_uuid: "baa8e36784a04ce29359e070394f97e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa8e36784a04ce29359e070394f97e7" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 } health_report { overall_health: UNKNOWN } } }
W20260928 06:41:05.104121 19528 tablet.cc:2369] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:41:05.107268 19542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:05.103229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43174 (local address 127.8.151.193:37789)
0928 06:41:05.103339 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:05.103343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:05.103378 (+ 35us) server_negotiation.cc:407] Connection header received
0928 06:41:05.103539 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:05.103542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:05.103600 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:05.103691 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:41:05.103933 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:05.104673 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:05.105766 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:05.106037 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:05.106117 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:05.106440 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:41:05.106849 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:41:05.106946 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:05.107009 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 06:41:05.107060 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 06:41:05.113914 19481 raft_consensus.cc:1261] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f [term 1 FOLLOWER]: Refusing update from remote peer baa8e36784a04ce29359e070394f97e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:41:05.113926 19347 raft_consensus.cc:1261] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4 [term 1 FOLLOWER]: Refusing update from remote peer baa8e36784a04ce29359e070394f97e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:41:05.114681 19552 consensus_queue.cc:1035] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dadae0a7023e4a6486f5c6ccf41622f4" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 35347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:41:05.114880 19555 consensus_queue.cc:1035] T a63bf9045a4b4534b4703eb99d76c823 P baa8e36784a04ce29359e070394f97e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e589246d2cab40348707ed497b91a52f" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 45149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:41:05.124794 19560 mvcc.cc:205] Tried to move back new op lower bound from 7334206116299022336 to 7334206116095197184. Current Snapshot: MvccSnapshot[applied={T|T < 7334206116299022336}]
I20260928 06:41:05.126688 19559 mvcc.cc:205] Tried to move back new op lower bound from 7334206116299022336 to 7334206116095197184. Current Snapshot: MvccSnapshot[applied={T|T < 7334206116299022336}]
I20260928 06:41:05.131096 19067 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:41:05.131326 19067 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:41:05.134176 19067 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6385de874ef34c958020e1a7e6163b71: Sending DeleteTablet for 3 replicas of tablet a63bf9045a4b4534b4703eb99d76c823
I20260928 06:41:05.135325 19327 tablet_service.cc:1514] Processing DeleteTablet for tablet a63bf9045a4b4534b4703eb99d76c823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:41:05 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:47444
I20260928 06:41:05.135823 19567 tablet_replica.cc:330] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: stopping tablet replica
I20260928 06:41:05.135897 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 19130
I20260928 06:41:05.135866 19193 tablet_service.cc:1514] Processing DeleteTablet for tablet a63bf9045a4b4534b4703eb99d76c823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:41:05 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:43170
I20260928 06:41:05.136162 19567 raft_consensus.cc:2229] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:41:05.136445 19567 raft_consensus.cc:2258] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:41:05.137584 19461 tablet_service.cc:1514] Processing DeleteTablet for tablet a63bf9045a4b4534b4703eb99d76c823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:41:05 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:58840
I20260928 06:41:05.138118 19567 ts_tablet_manager.cc:1905] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:41:05.140692 19567 ts_tablet_manager.cc:1918] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:41:05.140797 19567 log.cc:1196] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/a63bf9045a4b4534b4703eb99d76c823
I20260928 06:41:05.141140 19567 ts_tablet_manager.cc:1939] T a63bf9045a4b4534b4703eb99d76c823 P dadae0a7023e4a6486f5c6ccf41622f4: Deleting consensus metadata
W20260928 06:41:05.142094 19055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:37789 (error 108)
I20260928 06:41:05.142647 19055 catalog_manager.cc:4841] TS dadae0a7023e4a6486f5c6ccf41622f4 (127.8.151.194:35347): tablet a63bf9045a4b4534b4703eb99d76c823 (table test-table [id=3195070bab954cc79609795ccac4943b]) successfully deleted
I20260928 06:41:05.143071 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 19264
I20260928 06:41:05.143297 19538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:05.142964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:37789 (local address 127.0.0.1:43176)
0928 06:41:05.143094 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:05.143207 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:37789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 06:41:05.143548 19568 tablet_replica.cc:330] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: stopping tablet replica
I20260928 06:41:05.143872 19568 raft_consensus.cc:2229] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:41:05.144193 19568 raft_consensus.cc:2258] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 06:41:05.144264 19055 catalog_manager.cc:4584] TS baa8e36784a04ce29359e070394f97e7 (127.8.151.193:37789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a63bf9045a4b4534b4703eb99d76c823: Network error: Client connection negotiation failed: client connection to 127.8.151.193:37789: connect: Connection refused (error 111)
I20260928 06:41:05.145934 19568 ts_tablet_manager.cc:1905] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:41:05.148700 19568 ts_tablet_manager.cc:1918] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:41:05.148836 19568 log.cc:1196] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/a63bf9045a4b4534b4703eb99d76c823
I20260928 06:41:05.149103 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 19398
I20260928 06:41:05.149189 19568 ts_tablet_manager.cc:1939] T a63bf9045a4b4534b4703eb99d76c823 P e589246d2cab40348707ed497b91a52f: Deleting consensus metadata
I20260928 06:41:05.156935 19538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:05.156638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:45149 (local address 127.0.0.1:58842)
0928 06:41:05.156763 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:05.156825 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.195:45149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260928 06:41:05.157038 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 19036
2026-09-28T06:41:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2320 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:41:05 dist-test-slave-gtr8 krb5kdc[19572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:41:05 dist-test-slave-gtr8 krb5kdc[19572](info): set up 2 sockets
Sep 28 06:41:05 dist-test-slave-gtr8 krb5kdc[19572](info): commencing operation
krb5kdc: starting...
W20260928 06:41:06.128463 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:41:06 dist-test-slave-gtr8 krb5kdc[19572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:41:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:41:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.151.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.151.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/rangeradmin_127.8.151.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/rangeradmin_127.8.151.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.151.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.151.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/rangerlookup_127.8.151.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/rangerlookup_127.8.151.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.151.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.151.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/HTTP_127.8.151.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/HTTP_127.8.151.212@KRBTEST.COM.keytab.
I20260928 06:41:06.301906 8799 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYzMBXE/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/postgres -l logfile start
2026-09-28 06:41:08.931 UTC [19602] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 06:41:08.931 UTC [19602] LOG: listening on IPv4 address "127.8.151.212", port 53211
2026-09-28 06:41:08.935 UTC [19602] LOG: listening on Unix socket "/tmp/.s.PGSQL.53211"
2026-09-28 06:41:08.962 UTC [19605] LOG: database system was shut down at 2026-09-28 06:41:07 UTC
2026-09-28 06:41:08.966 UTC [19602] LOG: database system is ready to accept connections
I20260928 06:41:09.836232 8799 mini_postgres.cc:96] Postgres bound to 53211
2026-09-28 06:41:09.848 UTC [19613] FATAL: database "slave" does not exist
127.8.151.212:53211 - accepting connections
I20260928 06:41:09.849382 8799 mini_ranger.cc:161] Starting Ranger...
I20260928 06:41:09.865424 8799 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 06:41:10.347635 8799 mini_ranger.cc:87] Created ranger Postgres database
I20260928 06:41:10.347711 8799 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-admin
2026-09-28 06:41:10,759 [I] DB FLAVOR :POSTGRES
2026-09-28 06:41:10,759 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 06:41:10,759 [I] Checking connection..
2026-09-28 06:41:10,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 06:41:11,068 [I] Checking connection passed.
2026-09-28 06:41:11,068 [I] --------- Verifying version history table ---------
2026-09-28 06:41:11,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 06:41:11,361 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 06:41:11,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 06:41:11,642 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 06:41:11,642 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 06:41:11,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 06:41:11.927 UTC [19677] WARNING: there is no transaction in progress
2026-09-28 06:41:11,933 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 06:41:11,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 06:41:12,191 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 06:41:12,192 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 06:41:12,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 06:41:12,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 06:41:12,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gtr8.c.gcp-upstream.internal','N') ;"
2026-09-28 06:41:13,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 06:41:13,311 [I] Table x_portal_user does not exist in database ranger
2026-09-28 06:41:13,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 06:41:13,610 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 06:41:13,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 06:41:13,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 06:41:14,147 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 06:41:14,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 06:41:15.124 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.140 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.156 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.172 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.191 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.212 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.273 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.279 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.293 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.303 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.313 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.330 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.341 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.349 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.356 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:15.362 UTC [19785] WARNING: there is no transaction in progress
2026-09-28 06:41:16,000 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 06:41:16,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 06:41:16,313 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 06:41:16,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 06:41:16,627 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 06:41:16,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 06:41:16,902 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 06:41:16,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 06:41:17,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 06:41:17,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gtr8.c.gcp-upstream.internal';"
2026-09-28 06:41:17,726 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 06:41:17,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 06:41:18,001 [I] DB_PATCHES have already been applied
I20260928 06:41:18.008944 8799 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYzMBXE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 06:41:18.009011 8799 mini_ranger.cc:193] Using host: 127.8.151.212
I20260928 06:41:18.012375 8799 mini_ranger.cc:227] Ranger admin URL: http://127.8.151.212:45293
Sep 28, 2026 6:41:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 6:41:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 6:41:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 6:41:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 6:41:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 6:41:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 6:41:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 06:41:20.115617 8799 mini_ranger.cc:160] Time spent starting Ranger: real 10.266s user 0.007s sys 0.005s
Sep 28, 2026 6:41:45 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 6:41:45 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 6:41:45 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 6:41:45 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 6:41:47 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 06:41:47.866477 8799 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.151.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.151.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/rangerkms_127.8.151.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/rangerkms_127.8.151.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.151.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.151.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.151.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/HTTP_127.8.151.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.151.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/HTTP_127.8.151.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 06:41:47 dist-test-slave-gtr8 krb5kdc[19572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577707, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 06:41:47.972009 8799 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 06:41:47.985417 8799 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 06:41:48.940523 8799 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 06:41:48.940629 8799 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-kms
I20260928 06:41:48.940735 8799 mini_ranger_kms.cc:227] Using postgres at 127.8.151.212:53211
2026-09-28 06:41:49,757 [I] DB FLAVOR :POSTGRES
2026-09-28 06:41:49,757 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 06:41:49,757 [I] Checking connection
2026-09-28 06:41:49,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 06:41:50,054 [I] connection success
2026-09-28 06:41:50,055 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 06:41:50,055 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 06:41:50,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 06:41:50,319 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 06:41:50,319 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 06:41:50,319 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 06:41:50,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:53211/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 06:41:50,650 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 06:41:51.217486 8799 mini_ranger_kms.cc:310] Created kms service
I20260928 06:41:51.441574 8799 mini_ranger_kms.cc:326] Created kudu user
I20260928 06:41:51.486420 8799 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 06:41:51 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577707, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:41:51.965133 8799 mini_ranger_kms.cc:384] Added ranger policy
I20260928 06:41:51.965286 8799 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 06:41:51.965305 8799 mini_ranger_kms.cc:240] Using host: 127.8.151.212
I20260928 06:41:51.971628 8799 mini_ranger_kms.cc:276] Ranger KMS PID: 19967
I20260928 06:41:51.971772 8799 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.151.212:34317
Sep 28, 2026 6:41:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 6:41:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 6:41:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 6:41:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 6:41:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 6:41:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 06:41:54.115417 8799 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.143s user 0.042s sys 0.336s
I20260928 06:41:54.115540 8799 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 06:41:54.115640 8799 mini_ranger_kms.cc:401] 127.8.151.212:34317/kms/v1/keys
Sep 28 06:41:54 dist-test-slave-gtr8 krb5kdc[19572](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790577714, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.151.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 06:41:55 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577714, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.151.212@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
Sep 28 06:41:55 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577714, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.151.212@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
Sep 28 06:41:58 dist-test-slave-gtr8 krb5kdc[19572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:41:58.267390 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:36449
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:36449
--ranger_config_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:34317/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:41:58.429709 20017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:58.430018 20017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:58.430073 20017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:58.434163 20017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:41:58.434231 20017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:58.434257 20017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:58.434278 20017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:41:58.434322 20017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:41:58.441361 20017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.151.212:34317/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:36449
--ranger_config_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:36449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:58.442431 20017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:58.443687 20017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:58.452196 20023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:58.452297 20025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:58.452196 20022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:58.452584 20017 server_base.cc:1034] running on GCE node
I20260928 06:41:58.453394 20017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:58.454483 20017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:58.455637 20017 hybrid_clock.cc:648] HybridClock initialized: now 1790577718455617 us; error 35 us; skew 500 ppm
Sep 28 06:41:58 dist-test-slave-gtr8 krb5kdc[19572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577718, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:41:58.458317 20017 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:41:58.459261 20017 webserver.cc:460] Webserver started at http://127.8.151.254:40463/ using document root <none> and password file <none>
I20260928 06:41:58.459789 20017 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:58.459924 20017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:58.460145 20017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 06:41:58 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577718, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:41:58.515729 20017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "84cdbd570f214fbda3707b7ca21d6b57"
format_stamp: "Formatted at 2026-09-28 06:41:58 on dist-test-slave-gtr8"
server_key: "f0f4cdc16a225a9ff5a11797c34da0a7"
server_key_iv: "a4407838900778cb3aacd8025ea105f8"
server_key_version: "kuduclusterkey@0"
I20260928 06:41:58.516624 20017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "84cdbd570f214fbda3707b7ca21d6b57"
format_stamp: "Formatted at 2026-09-28 06:41:58 on dist-test-slave-gtr8"
server_key: "f0f4cdc16a225a9ff5a11797c34da0a7"
server_key_iv: "a4407838900778cb3aacd8025ea105f8"
server_key_version: "kuduclusterkey@0"
I20260928 06:41:58.521855 20017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 06:41:58.540436 20032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:58.541998 20017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:41:58.542146 20017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "84cdbd570f214fbda3707b7ca21d6b57"
format_stamp: "Formatted at 2026-09-28 06:41:58 on dist-test-slave-gtr8"
server_key: "f0f4cdc16a225a9ff5a11797c34da0a7"
server_key_iv: "a4407838900778cb3aacd8025ea105f8"
server_key_version: "kuduclusterkey@0"
I20260928 06:41:58.542276 20017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:58.560784 20017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:58.564495 20017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:58.564774 20017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:58.574011 20017 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:36449
I20260928 06:41:58.574198 20084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:36449 every 8 connection(s)
I20260928 06:41:58.575385 20017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:41:58.578895 20085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:41:58.581846 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 20017
I20260928 06:41:58.581974 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:41:58.582259 8799 external_mini_cluster.cc:1428] Setting key dadee7eb400870b5df8b3dbde9678a8d
I20260928 06:41:58.586038 20085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57: Bootstrap starting.
I20260928 06:41:58.589289 20085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57: Neither blocks nor log segments found. Creating new log.
I20260928 06:41:58.590289 20085 log.cc:824] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57: Log is configured to *not* fsync() on all Append() calls
I20260928 06:41:58.592628 20085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57: No bootstrap required, opened a new log
Sep 28 06:41:58 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:41:58.598093 20085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cdbd570f214fbda3707b7ca21d6b57" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 36449 } }
I20260928 06:41:58.598489 20085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:41:58.598603 20085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84cdbd570f214fbda3707b7ca21d6b57, State: Initialized, Role: FOLLOWER
I20260928 06:41:58.599179 20085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cdbd570f214fbda3707b7ca21d6b57" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 36449 } }
I20260928 06:41:58.599406 20085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:41:58.599502 20085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:41:58.599628 20085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:41:58.600685 20085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cdbd570f214fbda3707b7ca21d6b57" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 36449 } }
I20260928 06:41:58.601091 20085 leader_election.cc:302] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84cdbd570f214fbda3707b7ca21d6b57; no voters:
I20260928 06:41:58.601473 20085 leader_election.cc:288] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:41:58.602736 20085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:41:58.604925 20085 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 06:41:58.606156 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:58.584238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40106 (local address 127.8.151.254:36449)
0928 06:41:58.584839 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:58.584861 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:58.585193 (+ 332us) server_negotiation.cc:407] Connection header received
0928 06:41:58.586222 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:58.586247 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:58.586715 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:58.587060 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:58.588529 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:58.590127 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:58.591518 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:58.592242 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:58.592411 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:58.595829 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:58.595836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:58.595856 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:41:58.595890 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:58.602292 (+ 6402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:58.602834 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:58.602840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:58.602846 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:58.602985 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:58.603299 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:58.603302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:58.603304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:58.603843 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:58.604031 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:58.604794 (+ 763us) server_negotiation.cc:299] Negotiation successful
0928 06:41:58.605088 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":127,"threads_started":1}
I20260928 06:41:58.606403 20091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:41:58.610541 20091 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [term 1 LEADER]: Becoming Leader. State: Replica: 84cdbd570f214fbda3707b7ca21d6b57, State: Running, Role: LEADER
I20260928 06:41:58.611124 20091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cdbd570f214fbda3707b7ca21d6b57" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 36449 } }
I20260928 06:41:58.614705 20091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84cdbd570f214fbda3707b7ca21d6b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cdbd570f214fbda3707b7ca21d6b57" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 36449 } } }
I20260928 06:41:58.615192 20091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [sys.catalog]: This master's current role is: LEADER
I20260928 06:41:58.615046 20090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84cdbd570f214fbda3707b7ca21d6b57. Latest consensus state: current_term: 1 leader_uuid: "84cdbd570f214fbda3707b7ca21d6b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cdbd570f214fbda3707b7ca21d6b57" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 36449 } } }
I20260928 06:41:58.615593 20090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57 [sys.catalog]: This master's current role is: LEADER
I20260928 06:41:58.616745 20106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:41:58.620334 20106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:41:58.630852 20106 catalog_manager.cc:1347] Generated new cluster ID: d6358d6016494f2fa98e5bb1a75c1c4b
I20260928 06:41:58.630995 20106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:41:58.675100 20106 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:41:58.676685 20106 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:41:58.695690 20106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57: Generated new TSK 0
I20260928 06:41:58.696779 20106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:41:59.571902 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:34317/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:36449
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:41:59.719594 20130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:41:59.719852 20130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:41:59.719907 20130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:41:59.724269 20130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:41:59.724327 20130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:41:59.724403 20130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:41:59.730754 20130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.151.212:34317/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:36449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:41:59.731869 20130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:41:59.732875 20130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:41:59.742120 20135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:59.742226 20138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:41:59.742123 20136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:41:59.743355 20130 server_base.cc:1034] running on GCE node
I20260928 06:41:59.743922 20130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:41:59.744580 20130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:41:59.747949 20130 hybrid_clock.cc:648] HybridClock initialized: now 1790577719747885 us; error 71 us; skew 500 ppm
Sep 28 06:41:59 dist-test-slave-gtr8 krb5kdc[19572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577719, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:41:59.751590 20130 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:41:59.752619 20130 webserver.cc:460] Webserver started at http://127.8.151.193:35539/ using document root <none> and password file <none>
I20260928 06:41:59.755803 20130 fs_manager.cc:359] Metadata directory not provided
I20260928 06:41:59.755923 20130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:41:59.756176 20130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 06:41:59 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577719, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:41:59.772775 20130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "5cf6fbc2d59f481497681cb254f5d5d0"
format_stamp: "Formatted at 2026-09-28 06:41:59 on dist-test-slave-gtr8"
server_key: "2a5331e0ca009c8e08126ad33bb60b3e"
server_key_iv: "f6ba4c152a5cd950058ad9ab39082400"
server_key_version: "kuduclusterkey@0"
I20260928 06:41:59.773794 20130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "5cf6fbc2d59f481497681cb254f5d5d0"
format_stamp: "Formatted at 2026-09-28 06:41:59 on dist-test-slave-gtr8"
server_key: "2a5331e0ca009c8e08126ad33bb60b3e"
server_key_iv: "f6ba4c152a5cd950058ad9ab39082400"
server_key_version: "kuduclusterkey@0"
I20260928 06:41:59.784770 20130 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.000s sys 0.004s
I20260928 06:41:59.804739 20147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:59.809654 20130 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260928 06:41:59.809875 20130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "5cf6fbc2d59f481497681cb254f5d5d0"
format_stamp: "Formatted at 2026-09-28 06:41:59 on dist-test-slave-gtr8"
server_key: "2a5331e0ca009c8e08126ad33bb60b3e"
server_key_iv: "f6ba4c152a5cd950058ad9ab39082400"
server_key_version: "kuduclusterkey@0"
I20260928 06:41:59.810196 20130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:41:59.833508 20130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:41:59.844118 20130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:41:59.844408 20130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:41:59.845211 20130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:41:59.846287 20130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:41:59.846402 20130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:59.846539 20130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:41:59.846714 20130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:41:59.856681 20130 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:34811
I20260928 06:41:59.857829 20130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:41:59.860972 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 20130
I20260928 06:41:59.861092 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:41:59.861413 8799 external_mini_cluster.cc:1428] Setting key 00791bcae02ab6a4223840f9119c2114
I20260928 06:41:59.869036 20260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:34811 every 8 connection(s)
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
Sep 28 06:41:59 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577719, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:41:59.897508 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:59.871400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:36449 (local address 127.8.151.193:42313)
0928 06:41:59.876961 (+ 5561us) negotiation.cc:107] Waiting for socket to connect
0928 06:41:59.877026 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0928 06:41:59.880996 (+ 3970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:41:59.882010 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:41:59.882019 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:41:59.882353 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:41:59.883030 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:59.883049 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:59.884247 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:59.884250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:59.885634 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:41:59.885644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:41:59.886785 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:41:59.886789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:41:59.887007 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:59.887852 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:41:59.887874 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:41:59.892135 (+ 4261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:41:59.894269 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:41:59.894275 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:41:59.894292 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:41:59.894620 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:41:59.894940 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:41:59.894943 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:41:59.894946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:41:59.895120 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:41:59.895601 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:41:59.895608 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:41:59.895925 (+ 317us) client_negotiation.cc:769] Sending connection context
0928 06:41:59.896118 (+ 193us) client_negotiation.cc:240] Negotiation successful
0928 06:41:59.896435 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5289,"thread_start_us":145,"threads_started":1}
I20260928 06:41:59.898257 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:41:59.870811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:42313 (local address 127.8.151.254:36449)
0928 06:41:59.881473 (+ 10662us) server_negotiation.cc:206] Beginning negotiation
0928 06:41:59.881479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:41:59.881495 (+ 16us) server_negotiation.cc:407] Connection header received
0928 06:41:59.881539 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:41:59.881542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:41:59.881591 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:41:59.881676 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:41:59.883219 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:59.884101 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:59.886145 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:41:59.886518 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:41:59.886613 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:41:59.892344 (+ 5731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:41:59.892349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:41:59.892352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:41:59.892382 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:41:59.894139 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:59.894736 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:59.894739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:59.894741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:59.894826 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:41:59.895245 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:41:59.895248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:41:59.895250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:41:59.895464 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:41:59.895551 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 06:41:59.897971 (+ 2420us) server_negotiation.cc:299] Negotiation successful
0928 06:41:59.898097 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10569,"thread_start_us":114,"threads_started":1}
I20260928 06:41:59.904652 20261 heartbeater.cc:347] Connected to a master server at 127.8.151.254:36449
I20260928 06:41:59.904917 20261 heartbeater.cc:464] Registering TS with master...
I20260928 06:41:59.905526 20261 heartbeater.cc:511] Master 127.8.151.254:36449 requested a full tablet report, sending...
I20260928 06:41:59.907451 20049 ts_manager.cc:194] Registered new tserver with Master: 5cf6fbc2d59f481497681cb254f5d5d0 (127.8.151.193:34811)
I20260928 06:41:59.909014 20049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:42313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:41:59.917833 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:34317/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:36449
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 06:41:59 dist-test-slave-gtr8 krb5kdc[19572](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790577719, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 06:42:00.082870 20269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:00.083352 20269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:00.083473 20269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:00.089679 20269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:00.089814 20269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:00.089946 20269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:42:00.096177 20269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.151.212:34317/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:36449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:00.097656 20269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:00.098979 20269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:00.117707 20275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:00.117726 20276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:00.117707 20278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:00.119431 20269 server_base.cc:1034] running on GCE node
I20260928 06:42:00.119915 20269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:00.120712 20269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:00.121871 20269 hybrid_clock.cc:648] HybridClock initialized: now 1790577720121806 us; error 80 us; skew 500 ppm
Sep 28 06:42:00 dist-test-slave-gtr8 krb5kdc[19572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577720, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:00.125070 20269 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:42:00.126019 20269 webserver.cc:460] Webserver started at http://127.8.151.194:39195/ using document root <none> and password file <none>
I20260928 06:42:00.126617 20269 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:00.126753 20269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:42:00.127038 20269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 06:42:00 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577720, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:00.147969 20269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "0d32a21a1dc34006acaf0281047f1fe3"
format_stamp: "Formatted at 2026-09-28 06:42:00 on dist-test-slave-gtr8"
server_key: "5135a175b2c48f69a35ca708f9e1b9b3"
server_key_iv: "caa79eac24b9bc5ca3a878ad88988cf4"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:00.151050 20269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "0d32a21a1dc34006acaf0281047f1fe3"
format_stamp: "Formatted at 2026-09-28 06:42:00 on dist-test-slave-gtr8"
server_key: "5135a175b2c48f69a35ca708f9e1b9b3"
server_key_iv: "caa79eac24b9bc5ca3a878ad88988cf4"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:00.157735 20269 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260928 06:42:00.174286 20285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:00.176060 20269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 06:42:00.176203 20269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "0d32a21a1dc34006acaf0281047f1fe3"
format_stamp: "Formatted at 2026-09-28 06:42:00 on dist-test-slave-gtr8"
server_key: "5135a175b2c48f69a35ca708f9e1b9b3"
server_key_iv: "caa79eac24b9bc5ca3a878ad88988cf4"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:00.176337 20269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:00.228610 20269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:00.234108 20269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:00.234390 20269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:00.235229 20269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:42:00.236346 20269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:42:00.236454 20269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:00.236552 20269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:42:00.236618 20269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:00.251215 20269 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:41047
I20260928 06:42:00.252537 20269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:42:00.253710 20398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:41047 every 8 connection(s)
I20260928 06:42:00.254447 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 20269
I20260928 06:42:00.254566 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:42:00.254906 8799 external_mini_cluster.cc:1428] Setting key 7b1f8b5f98eea54389768d22d3cb9399
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Sep 28 06:42:00 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577720, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:00.285316 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.261263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:39187 (local address 127.8.151.254:36449)
0928 06:42:00.261414 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:00.261419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:00.264593 (+ 3174us) server_negotiation.cc:407] Connection header received
0928 06:42:00.265786 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:00.265790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:00.265858 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:00.265948 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:00.271825 (+ 5877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.272732 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.274649 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.275010 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.275094 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.280111 (+ 5017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:00.280115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:00.280118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:00.280150 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:00.282344 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:00.283293 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:00.283297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:00.283299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:00.283393 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:00.283832 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:00.283836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:00.283838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:00.284083 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:00.284175 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:00.284955 (+ 780us) server_negotiation.cc:299] Negotiation successful
0928 06:42:00.285108 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 06:42:00.287310 20402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.260180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:36449 (local address 127.8.151.194:39187)
0928 06:42:00.264416 (+ 4236us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:00.264480 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:00.265551 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:00.266260 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:00.266270 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:00.266619 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:00.271644 (+ 5025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.271670 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.272911 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.272915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.274484 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.274496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.275367 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.275372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.275581 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.278064 (+ 2483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:00.278088 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:00.279875 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:00.282651 (+ 2776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:00.282657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:00.282674 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:00.283169 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:00.283502 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:00.283505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:00.283507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:00.283702 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:00.284250 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:00.284257 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:00.284624 (+ 367us) client_negotiation.cc:769] Sending connection context
0928 06:42:00.285815 (+ 1191us) client_negotiation.cc:240] Negotiation successful
0928 06:42:00.286184 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3994,"thread_start_us":126,"threads_started":1}
I20260928 06:42:00.288865 20399 heartbeater.cc:347] Connected to a master server at 127.8.151.254:36449
I20260928 06:42:00.289103 20399 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:00.289664 20399 heartbeater.cc:511] Master 127.8.151.254:36449 requested a full tablet report, sending...
I20260928 06:42:00.290802 20049 ts_manager.cc:194] Registered new tserver with Master: 0d32a21a1dc34006acaf0281047f1fe3 (127.8.151.194:41047)
I20260928 06:42:00.291622 20049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:39187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:42:00.310073 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:34317/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:36449
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:42:00.458120 20404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:00.458588 20404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:00.458704 20404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:00.464663 20404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:00.464798 20404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:00.464923 20404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:42:00.472872 20404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.151.212:34317/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:36449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:00.474406 20404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:00.475839 20404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:00.488215 20410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:00.488636 20412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:00.488924 20409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:00.490176 20404 server_base.cc:1034] running on GCE node
I20260928 06:42:00.490648 20404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:00.491469 20404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:00.509266 20404 hybrid_clock.cc:648] HybridClock initialized: now 1790577720509165 us; error 122 us; skew 500 ppm
Sep 28 06:42:00 dist-test-slave-gtr8 krb5kdc[19572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577720, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:00.513662 20404 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:42:00.514990 20404 webserver.cc:460] Webserver started at http://127.8.151.195:39165/ using document root <none> and password file <none>
I20260928 06:42:00.515857 20404 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:00.516023 20404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:42:00.516328 20404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 06:42:00 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577720, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:00.533036 20404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "9108cf20f37c48f3ac6251aadf28b00a"
format_stamp: "Formatted at 2026-09-28 06:42:00 on dist-test-slave-gtr8"
server_key: "3e10c36aca76efd284fecdd48bccd0e8"
server_key_iv: "64c225f7e8b378e1d370e158f975225f"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:00.533947 20404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "9108cf20f37c48f3ac6251aadf28b00a"
format_stamp: "Formatted at 2026-09-28 06:42:00 on dist-test-slave-gtr8"
server_key: "3e10c36aca76efd284fecdd48bccd0e8"
server_key_iv: "64c225f7e8b378e1d370e158f975225f"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:00.540627 20404 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260928 06:42:00.559875 20419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:00.561620 20404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260928 06:42:00.561748 20404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "9108cf20f37c48f3ac6251aadf28b00a"
format_stamp: "Formatted at 2026-09-28 06:42:00 on dist-test-slave-gtr8"
server_key: "3e10c36aca76efd284fecdd48bccd0e8"
server_key_iv: "64c225f7e8b378e1d370e158f975225f"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:00.561888 20404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:00.587643 20404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:00.592540 20404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:00.592842 20404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:00.593922 20404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:42:00.595366 20404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:42:00.595501 20404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:00.595736 20404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:42:00.595854 20404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:00.608843 20404 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:36021
I20260928 06:42:00.608911 20532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:36021 every 8 connection(s)
I20260928 06:42:00.610242 20404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:42:00.614081 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 20404
I20260928 06:42:00.614202 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:42:00.614506 8799 external_mini_cluster.cc:1428] Setting key 143ae940e05cc5f8aed4e7fea1e6fac2
Sep 28 06:42:00 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577719, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
Sep 28 06:42:00 dist-test-slave-gtr8 krb5kdc[19572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577720, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:00.631536 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.613227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:54663 (local address 127.8.151.254:36449)
0928 06:42:00.613384 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:00.613389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:00.617835 (+ 4446us) server_negotiation.cc:407] Connection header received
0928 06:42:00.619058 (+ 1223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:00.619062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:00.619116 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:00.619194 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:00.621456 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.622267 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.624166 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.624499 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.624581 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.627496 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:00.627500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:00.627502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:00.627528 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:00.629115 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:00.629716 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:00.629720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:00.629721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:00.629820 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:00.630232 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:00.630235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:00.630237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:00.630461 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:00.630543 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:00.631253 (+ 710us) server_negotiation.cc:299] Negotiation successful
0928 06:42:00.631372 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 06:42:00.633118 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.613709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:36449 (local address 127.8.151.195:54663)
0928 06:42:00.617669 (+ 3960us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:00.617730 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:00.618814 (+ 1084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:00.620162 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:00.620172 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:00.620520 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:00.621290 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.621311 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.622430 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.622433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.624024 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.624036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.624782 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.624786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.625041 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.625699 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:00.625721 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:00.627339 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:00.629253 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:00.629259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:00.629273 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:00.629601 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:00.629942 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:00.629945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:00.629947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:00.630121 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:00.630612 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:00.630618 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:00.630936 (+ 318us) client_negotiation.cc:769] Sending connection context
0928 06:42:00.631701 (+ 765us) client_negotiation.cc:240] Negotiation successful
0928 06:42:00.632058 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3702,"thread_start_us":137,"threads_started":1}
I20260928 06:42:00.634474 20533 heartbeater.cc:347] Connected to a master server at 127.8.151.254:36449
I20260928 06:42:00.634701 20533 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:00.635298 20533 heartbeater.cc:511] Master 127.8.151.254:36449 requested a full tablet report, sending...
I20260928 06:42:00.636370 20049 ts_manager.cc:194] Registered new tserver with Master: 9108cf20f37c48f3ac6251aadf28b00a (127.8.151.195:36021)
I20260928 06:42:00.637108 20049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:54663
I20260928 06:42:00.642781 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:42:00.655857 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.644955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.8.151.254:36449)
0928 06:42:00.645066 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:00.645070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:00.645631 (+ 561us) server_negotiation.cc:407] Connection header received
0928 06:42:00.645791 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:00.645794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:00.645838 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:00.645882 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:00.646848 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.647620 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.648783 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.649066 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.649129 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.651928 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:00.651932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:00.651935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:00.651961 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:00.653466 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:00.654111 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:00.654115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:00.654116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:00.654196 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:00.654760 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:00.654763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:00.654765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:00.654977 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:00.655074 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:00.655579 (+ 505us) server_negotiation.cc:299] Negotiation successful
0928 06:42:00.655697 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 06:42:00.659925 20049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:42:00.664006 20049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:42:00.697508 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.691672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:41047 (local address 127.0.0.1:50826)
0928 06:42:00.691920 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:00.691933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:00.692014 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:00.692850 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:00.692856 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:00.692879 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:42:00.693019 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.693026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.694509 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.694512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.695849 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.695859 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.696961 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.696965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.697086 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.697132 (+ 46us) client_negotiation.cc:769] Sending connection context
0928 06:42:00.697221 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 06:42:00.697336 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":115,"threads_started":1}
I20260928 06:42:00.698148 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.692098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.8.151.194:41047)
0928 06:42:00.692328 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:00.692344 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:00.692367 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:42:00.692443 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:00.692448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:00.692669 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:00.693178 (+ 509us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:42:00.693312 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.694380 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.696011 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.696804 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.697690 (+ 886us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.697856 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:00.697914 (+ 58us) server_negotiation.cc:299] Negotiation successful
0928 06:42:00.698010 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":72,"threads_started":1}
I20260928 06:42:00.702618 20333 tablet_service.cc:1467] Processing CreateTablet for tablet 767071ea1464455aa9ab347c0fda2548 (DEFAULT_TABLE table=test-table [id=1bed4fe393874ae6b3334c763669aff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:42:00.704636 20333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 767071ea1464455aa9ab347c0fda2548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:42:00.720676 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.688660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:34811 (local address 127.0.0.1:59808)
0928 06:42:00.689134 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:00.689181 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:00.689298 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:00.696918 (+ 7620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:00.696921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:00.696938 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:42:00.697064 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.697070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.712640 (+ 15570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.712644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.713876 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.713886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.720216 (+ 6330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.720220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.720358 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.720386 (+ 28us) client_negotiation.cc:769] Sending connection context
0928 06:42:00.720437 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 06:42:00.720516 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":11,"thread_start_us":217,"threads_started":1}
I20260928 06:42:00.721660 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.689703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.8.151.193:34811)
0928 06:42:00.696462 (+ 6759us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:00.696470 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:00.696495 (+ 25us) server_negotiation.cc:407] Connection header received
0928 06:42:00.696567 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:00.696572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:00.696766 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:00.697233 (+ 467us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:42:00.697377 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.698507 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.714867 (+ 16360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.717637 (+ 2770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.721176 (+ 3539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.721352 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:00.721409 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 06:42:00.721516 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6648,"thread_start_us":74,"threads_started":1}
I20260928 06:42:00.723371 20547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.690891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:36021 (local address 127.0.0.1:41220)
0928 06:42:00.691207 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:00.691220 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:00.691307 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:00.712403 (+ 21096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:00.712407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:00.712429 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:42:00.712531 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.712537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.717421 (+ 4884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.717425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.718677 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.718688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.722929 (+ 4241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.722932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.723031 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.723057 (+ 26us) client_negotiation.cc:769] Sending connection context
0928 06:42:00.723157 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 06:42:00.723230 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":170,"threads_started":1}
I20260928 06:42:00.731712 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.691076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.8.151.195:36021)
0928 06:42:00.698783 (+ 7707us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:00.698794 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:00.698816 (+ 22us) server_negotiation.cc:407] Connection header received
0928 06:42:00.698886 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:00.698891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:00.699071 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:00.712440 (+ 13369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:42:00.714014 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.717310 (+ 3296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.721929 (+ 4619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.722811 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.722920 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.723051 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:00.731391 (+ 8340us) server_negotiation.cc:299] Negotiation successful
0928 06:42:00.731538 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7600,"thread_start_us":130,"threads_started":1}
I20260928 06:42:00.734536 20195 tablet_service.cc:1467] Processing CreateTablet for tablet 767071ea1464455aa9ab347c0fda2548 (DEFAULT_TABLE table=test-table [id=1bed4fe393874ae6b3334c763669aff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:42:00.734537 20460 tablet_service.cc:1467] Processing CreateTablet for tablet 767071ea1464455aa9ab347c0fda2548 (DEFAULT_TABLE table=test-table [id=1bed4fe393874ae6b3334c763669aff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:42:00.735771 20460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 767071ea1464455aa9ab347c0fda2548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:42:00.735771 20195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 767071ea1464455aa9ab347c0fda2548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:42:00.744375 20551 tablet_bootstrap.cc:492] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3: Bootstrap starting.
I20260928 06:42:00.747865 20551 tablet_bootstrap.cc:654] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3: Neither blocks nor log segments found. Creating new log.
I20260928 06:42:00.749502 20551 log.cc:824] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:00.750037 20552 tablet_bootstrap.cc:492] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0: Bootstrap starting.
I20260928 06:42:00.751977 20553 tablet_bootstrap.cc:492] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: Bootstrap starting.
I20260928 06:42:00.754824 20553 tablet_bootstrap.cc:654] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: Neither blocks nor log segments found. Creating new log.
I20260928 06:42:00.754822 20552 tablet_bootstrap.cc:654] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0: Neither blocks nor log segments found. Creating new log.
I20260928 06:42:00.755735 20553 log.cc:824] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:00.763875 20551 tablet_bootstrap.cc:492] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3: No bootstrap required, opened a new log
I20260928 06:42:00.764169 20551 ts_tablet_manager.cc:1397] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.000s
I20260928 06:42:00.764428 20552 log.cc:824] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:00.769690 20551 raft_consensus.cc:348] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 34811 } } peers { permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 } } peers { permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 } }
I20260928 06:42:00.770112 20551 raft_consensus.cc:374] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:42:00.770674 20551 raft_consensus.cc:729] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d32a21a1dc34006acaf0281047f1fe3, State: Initialized, Role: FOLLOWER
I20260928 06:42:00.771688 20551 consensus_queue.cc:260] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 34811 } } peers { permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 } } peers { permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 } }
I20260928 06:42:00.773046 20553 tablet_bootstrap.cc:492] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: No bootstrap required, opened a new log
I20260928 06:42:00.773319 20553 ts_tablet_manager.cc:1397] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.005s
I20260928 06:42:00.777433 20553 raft_consensus.cc:348] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 34811 } } peers { permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 } } peers { permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 } }
I20260928 06:42:00.777808 20553 raft_consensus.cc:374] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:42:00.777902 20553 raft_consensus.cc:729] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9108cf20f37c48f3ac6251aadf28b00a, State: Initialized, Role: FOLLOWER
I20260928 06:42:00.778476 20553 consensus_queue.cc:260] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 34811 } } peers { permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 } } peers { permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 } }
I20260928 06:42:00.778803 20552 tablet_bootstrap.cc:492] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0: No bootstrap required, opened a new log
I20260928 06:42:00.779060 20552 ts_tablet_manager.cc:1397] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.004s
I20260928 06:42:00.779855 20553 ts_tablet_manager.cc:1428] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:42:00.780267 20533 heartbeater.cc:503] Master 127.8.151.254:36449 was elected leader, sending a full tablet report...
I20260928 06:42:00.788908 20552 raft_consensus.cc:348] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 34811 } } peers { permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 } } peers { permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 } }
I20260928 06:42:00.790343 20551 ts_tablet_manager.cc:1428] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260928 06:42:00.790758 20399 heartbeater.cc:503] Master 127.8.151.254:36449 was elected leader, sending a full tablet report...
I20260928 06:42:00.793069 20552 raft_consensus.cc:374] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:42:00.793202 20552 raft_consensus.cc:729] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cf6fbc2d59f481497681cb254f5d5d0, State: Initialized, Role: FOLLOWER
I20260928 06:42:00.793779 20552 consensus_queue.cc:260] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 34811 } } peers { permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 } } peers { permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 } }
I20260928 06:42:00.795048 20552 ts_tablet_manager.cc:1428] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0: Time spent starting tablet: real 0.016s user 0.002s sys 0.004s
I20260928 06:42:00.795492 20261 heartbeater.cc:503] Master 127.8.151.254:36449 was elected leader, sending a full tablet report...
W20260928 06:42:00.861757 20534 tablet.cc:2369] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 06:42:00.873541 20262 tablet.cc:2369] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:42:00.899484 20559 raft_consensus.cc:482] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:42:00.899706 20559 raft_consensus.cc:504] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 34811 } } peers { permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 } } peers { permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 } }
I20260928 06:42:00.901139 20559 leader_election.cc:288] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9108cf20f37c48f3ac6251aadf28b00a (127.8.151.195:36021), 0d32a21a1dc34006acaf0281047f1fe3 (127.8.151.194:41047)
I20260928 06:42:00.908550 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.902535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:52305 (local address 127.8.151.195:36021)
0928 06:42:00.902652 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:00.902656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:00.904515 (+ 1859us) server_negotiation.cc:407] Connection header received
0928 06:42:00.904698 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:00.904701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:00.904754 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:00.904798 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:42:00.905164 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.905866 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.907260 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.907861 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.907949 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.908001 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:00.908322 (+ 321us) server_negotiation.cc:299] Negotiation successful
0928 06:42:00.908413 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:42:00.909153 20560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.902423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:36021 (local address 127.8.151.193:52305)
0928 06:42:00.904432 (+ 2009us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:00.904446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:00.904578 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:00.904924 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:00.904927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:00.904943 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:42:00.905040 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.905045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.906007 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.906009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.907105 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.907115 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.908102 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.908104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.908196 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.908217 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 06:42:00.908929 (+ 712us) client_negotiation.cc:240] Negotiation successful
0928 06:42:00.909002 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1937,"thread_start_us":107,"threads_started":1}
I20260928 06:42:00.909900 20472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "767071ea1464455aa9ab347c0fda2548" candidate_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9108cf20f37c48f3ac6251aadf28b00a" is_pre_election: true
I20260928 06:42:00.910219 20472 raft_consensus.cc:2454] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cf6fbc2d59f481497681cb254f5d5d0 in term 0.
I20260928 06:42:00.910811 20151 leader_election.cc:302] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf6fbc2d59f481497681cb254f5d5d0, 9108cf20f37c48f3ac6251aadf28b00a; no voters:
I20260928 06:42:00.911103 20559 raft_consensus.cc:2781] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:42:00.911193 20559 raft_consensus.cc:482] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:42:00.911237 20559 raft_consensus.cc:3037] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:42:00.911914 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.902162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:44139 (local address 127.8.151.194:41047)
0928 06:42:00.902288 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:00.902292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:00.902898 (+ 606us) server_negotiation.cc:407] Connection header received
0928 06:42:00.903120 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:00.903124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:00.903198 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:00.903249 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:42:00.903640 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.904421 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.910662 (+ 6241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.911284 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.911402 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.911450 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:00.911711 (+ 261us) server_negotiation.cc:299] Negotiation successful
0928 06:42:00.911788 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:42:00.912230 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.901622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:41047 (local address 127.8.151.193:44139)
0928 06:42:00.902800 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:00.902813 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:00.902983 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:00.903391 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:00.903393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:00.903412 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:42:00.903514 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.903521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.904560 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.904563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.905778 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:00.905788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.911505 (+ 5717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.911509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:00.911584 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.911605 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 06:42:00.912037 (+ 432us) client_negotiation.cc:240] Negotiation successful
0928 06:42:00.912109 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"thread_start_us":129,"threads_started":1}
I20260928 06:42:00.912549 20559 raft_consensus.cc:504] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 34811 } } peers { permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 } } peers { permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 } }
I20260928 06:42:00.912920 20559 leader_election.cc:288] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9108cf20f37c48f3ac6251aadf28b00a (127.8.151.195:36021), 0d32a21a1dc34006acaf0281047f1fe3 (127.8.151.194:41047)
I20260928 06:42:00.913800 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "767071ea1464455aa9ab347c0fda2548" candidate_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d32a21a1dc34006acaf0281047f1fe3" is_pre_election: true
I20260928 06:42:00.914088 20346 raft_consensus.cc:2454] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cf6fbc2d59f481497681cb254f5d5d0 in term 0.
I20260928 06:42:00.914434 20352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "767071ea1464455aa9ab347c0fda2548" candidate_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d32a21a1dc34006acaf0281047f1fe3"
I20260928 06:42:00.914541 20352 raft_consensus.cc:3037] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:42:00.914853 20472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "767071ea1464455aa9ab347c0fda2548" candidate_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9108cf20f37c48f3ac6251aadf28b00a"
I20260928 06:42:00.914992 20472 raft_consensus.cc:3037] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:42:00.915750 20352 raft_consensus.cc:2454] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cf6fbc2d59f481497681cb254f5d5d0 in term 1.
I20260928 06:42:00.916442 20472 raft_consensus.cc:2454] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cf6fbc2d59f481497681cb254f5d5d0 in term 1.
I20260928 06:42:00.917001 20150 leader_election.cc:302] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d32a21a1dc34006acaf0281047f1fe3, 5cf6fbc2d59f481497681cb254f5d5d0; no voters:
I20260928 06:42:00.917330 20559 raft_consensus.cc:2781] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:42:00.917568 20559 raft_consensus.cc:686] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 5cf6fbc2d59f481497681cb254f5d5d0, State: Running, Role: LEADER
I20260928 06:42:00.917910 20559 consensus_queue.cc:237] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 34811 } } peers { permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 } } peers { permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 } }
I20260928 06:42:00.921312 20048 catalog_manager.cc:5495] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cf6fbc2d59f481497681cb254f5d5d0 (127.8.151.193). New cstate: current_term: 1 leader_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf6fbc2d59f481497681cb254f5d5d0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 34811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 } health_report { overall_health: UNKNOWN } } }
I20260928 06:42:00.935617 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:00.930437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59824 (local address 127.8.151.193:34811)
0928 06:42:00.930890 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:00.930895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:00.930909 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:42:00.930980 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:00.930983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:00.931040 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:00.931170 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:42:00.931457 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.932244 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.933531 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:00.933871 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:00.933960 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:00.934352 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:42:00.934860 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:42:00.934969 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:00.935394 (+ 425us) server_negotiation.cc:299] Negotiation successful
0928 06:42:00.935453 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367}
I20260928 06:42:00.942987 20352 raft_consensus.cc:1261] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3 [term 1 FOLLOWER]: Refusing update from remote peer 5cf6fbc2d59f481497681cb254f5d5d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:42:00.943843 20559 consensus_queue.cc:1035] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d32a21a1dc34006acaf0281047f1fe3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:42:00.950870 20472 raft_consensus.cc:1261] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a [term 1 FOLLOWER]: Refusing update from remote peer 5cf6fbc2d59f481497681cb254f5d5d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:42:00.951597 20559 consensus_queue.cc:1035] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9108cf20f37c48f3ac6251aadf28b00a" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 36021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:42:00.967913 20565 mvcc.cc:205] Tried to move back new op lower bound from 7334206344973799424 to 7334206344883187712. Current Snapshot: MvccSnapshot[applied={T|T < 7334206344973799424}]
I20260928 06:42:00.974725 20566 mvcc.cc:205] Tried to move back new op lower bound from 7334206344973799424 to 7334206344883187712. Current Snapshot: MvccSnapshot[applied={T|T < 7334206344973799424}]
I20260928 06:42:00.974985 20568 mvcc.cc:205] Tried to move back new op lower bound from 7334206344973799424 to 7334206344883187712. Current Snapshot: MvccSnapshot[applied={T|T < 7334206344973799424}]
I20260928 06:42:00.978463 20048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:42:00.978634 20048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:42:00.981029 20048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84cdbd570f214fbda3707b7ca21d6b57: Sending DeleteTablet for 3 replicas of tablet 767071ea1464455aa9ab347c0fda2548
I20260928 06:42:00.982105 20333 tablet_service.cc:1514] Processing DeleteTablet for tablet 767071ea1464455aa9ab347c0fda2548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:42:00 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:50826
I20260928 06:42:00.982360 20460 tablet_service.cc:1514] Processing DeleteTablet for tablet 767071ea1464455aa9ab347c0fda2548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:42:00 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:41220
I20260928 06:42:00.982954 20195 tablet_service.cc:1514] Processing DeleteTablet for tablet 767071ea1464455aa9ab347c0fda2548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:42:00 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:59808
I20260928 06:42:00.983415 20576 tablet_replica.cc:330] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0: stopping tablet replica
I20260928 06:42:00.983703 20576 raft_consensus.cc:2229] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:42:00.984077 20576 raft_consensus.cc:2258] T 767071ea1464455aa9ab347c0fda2548 P 5cf6fbc2d59f481497681cb254f5d5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:42:00.984179 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 20130
I20260928 06:42:00.993460 20575 tablet_replica.cc:330] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: stopping tablet replica
I20260928 06:42:00.993901 20575 raft_consensus.cc:2229] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:42:00.994355 20575 raft_consensus.cc:2258] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:42:00.996208 20575 ts_tablet_manager.cc:1905] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:42:00.996874 20574 tablet_replica.cc:330] T 767071ea1464455aa9ab347c0fda2548 P 0d32a21a1dc34006acaf0281047f1fe3: stopping tablet replica
W20260928 06:42:00.996903 20033 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:34811 (error 108)
W20260928 06:42:00.998111 20547 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:42:00.997671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:34811 (local address 127.0.0.1:59840)
0928 06:42:00.997797 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:00.997813 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:00.997917 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:00.998007 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:34811: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 06:42:00.998356 20033 catalog_manager.cc:4584] TS 5cf6fbc2d59f481497681cb254f5d5d0 (127.8.151.193:34811): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 767071ea1464455aa9ab347c0fda2548: Network error: Client connection negotiation failed: client connection to 127.8.151.193:34811: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260928 06:42:00.998740 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 20269
I20260928 06:42:01.002610 20575 ts_tablet_manager.cc:1918] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:42:01.002759 20575 log.cc:1196] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/767071ea1464455aa9ab347c0fda2548
I20260928 06:42:01.003222 20575 ts_tablet_manager.cc:1939] T 767071ea1464455aa9ab347c0fda2548 P 9108cf20f37c48f3ac6251aadf28b00a: Deleting consensus metadata
I20260928 06:42:01.004650 20036 catalog_manager.cc:4841] TS 9108cf20f37c48f3ac6251aadf28b00a (127.8.151.195:36021): tablet 767071ea1464455aa9ab347c0fda2548 (table test-table [id=1bed4fe393874ae6b3334c763669aff4]) successfully deleted
I20260928 06:42:01.013746 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 20404
I20260928 06:42:01.013991 20547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:01.013646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:41047 (local address 127.0.0.1:50830)
0928 06:42:01.013820 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:01.013906 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.194:41047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260928 06:42:01.020689 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 20017
I20260928 06:42:01.027881 8799 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 06:42:01.362316 8799 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 06:42:01.362483 8799 mini_ranger.cc:66] Stopping Ranger...
I20260928 06:42:01.697072 8799 mini_ranger.cc:68] Stopped Ranger
2026-09-28 06:42:01.697 UTC [19602] LOG: received smart shutdown request
2026-09-28 06:42:01.701 UTC [19602] LOG: background worker "logical replication launcher" (PID 19611) exited with exit code 1
2026-09-28 06:42:01.701 UTC [19606] LOG: shutting down
2026-09-28 06:42:01.812 UTC [19602] LOG: database system is shut down
2026-09-28T06:42:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (56788 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:42:01 dist-test-slave-gtr8 krb5kdc[20587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:42:01 dist-test-slave-gtr8 krb5kdc[20587](info): set up 2 sockets
Sep 28 06:42:01 dist-test-slave-gtr8 krb5kdc[20587](info): commencing operation
krb5kdc: starting...
W20260928 06:42:02.907590 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:42:02 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:42:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:42:02Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.151.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.151.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/rangeradmin_127.8.151.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/rangeradmin_127.8.151.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.151.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.151.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/rangerlookup_127.8.151.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/rangerlookup_127.8.151.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.151.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.151.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/HTTP_127.8.151.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/HTTP_127.8.151.212@KRBTEST.COM.keytab.
I20260928 06:42:03.074237 8799 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYzMBXE/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/postgres -l logfile start
2026-09-28 06:42:05.575 UTC [20617] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 06:42:05.575 UTC [20617] LOG: listening on IPv4 address "127.8.151.212", port 37931
2026-09-28 06:42:05.579 UTC [20617] LOG: listening on Unix socket "/tmp/.s.PGSQL.37931"
2026-09-28 06:42:05.598 UTC [20620] LOG: database system was shut down at 2026-09-28 06:42:04 UTC
2026-09-28 06:42:05.601 UTC [20617] LOG: database system is ready to accept connections
I20260928 06:42:06.479154 8799 mini_postgres.cc:96] Postgres bound to 37931
2026-09-28 06:42:06.485 UTC [20628] FATAL: database "slave" does not exist
127.8.151.212:37931 - accepting connections
I20260928 06:42:06.486003 8799 mini_ranger.cc:161] Starting Ranger...
I20260928 06:42:06.499876 8799 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 06:42:07.038185 8799 mini_ranger.cc:87] Created ranger Postgres database
I20260928 06:42:07.038249 8799 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-admin
2026-09-28 06:42:07,316 [I] DB FLAVOR :POSTGRES
2026-09-28 06:42:07,317 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 06:42:07,317 [I] Checking connection..
2026-09-28 06:42:07,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 06:42:07,581 [I] Checking connection passed.
2026-09-28 06:42:07,581 [I] --------- Verifying version history table ---------
2026-09-28 06:42:07,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 06:42:07,852 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 06:42:07,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 06:42:08,128 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 06:42:08,129 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 06:42:08,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 06:42:08.415 UTC [20692] WARNING: there is no transaction in progress
2026-09-28 06:42:08,421 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 06:42:08,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 06:42:08,713 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 06:42:08,714 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 06:42:08,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 06:42:08,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 06:42:09,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gtr8.c.gcp-upstream.internal','N') ;"
2026-09-28 06:42:09,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 06:42:09,795 [I] Table x_portal_user does not exist in database ranger
2026-09-28 06:42:09,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 06:42:10,062 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 06:42:10,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 06:42:10,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 06:42:10,633 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 06:42:10,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 06:42:11.676 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.693 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.710 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.725 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.742 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.759 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.815 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.821 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.836 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.847 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.858 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.875 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.885 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.892 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.899 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:11.907 UTC [20800] WARNING: there is no transaction in progress
2026-09-28 06:42:12,566 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 06:42:12,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 06:42:12,810 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 06:42:12,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 06:42:13,092 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 06:42:13,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 06:42:13,372 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 06:42:13,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 06:42:13,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 06:42:13,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gtr8.c.gcp-upstream.internal';"
2026-09-28 06:42:14,210 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 06:42:14,210 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 06:42:14,496 [I] DB_PATCHES have already been applied
I20260928 06:42:14.502044 8799 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYzMBXE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 06:42:14.502116 8799 mini_ranger.cc:193] Using host: 127.8.151.212
I20260928 06:42:14.505714 8799 mini_ranger.cc:227] Ranger admin URL: http://127.8.151.212:48311
Sep 28, 2026 6:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 6:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 6:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 6:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 6:42:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 6:42:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 6:42:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 06:42:16.665449 8799 mini_ranger.cc:160] Time spent starting Ranger: real 10.179s user 0.000s sys 0.011s
Sep 28, 2026 6:42:39 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 6:42:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 28, 2026 6:42:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 6:42:39 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 6:42:41 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 06:42:41.795925 8799 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.151.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.151.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/rangerkms_127.8.151.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.151.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/rangerkms_127.8.151.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.151.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.151.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.151.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/HTTP_127.8.151.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.151.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/HTTP_127.8.151.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 06:42:41 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577761, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 06:42:41.900589 8799 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 06:42:41.914366 8799 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 06:42:42.721434 8799 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 06:42:42.721511 8799 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-kms
I20260928 06:42:42.721529 8799 mini_ranger_kms.cc:227] Using postgres at 127.8.151.212:37931
2026-09-28 06:42:42,911 [I] DB FLAVOR :POSTGRES
2026-09-28 06:42:42,911 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 06:42:42,911 [I] Checking connection
2026-09-28 06:42:42,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 06:42:43,170 [I] connection success
2026-09-28 06:42:43,170 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 06:42:43,170 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 06:42:43,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 06:42:43,429 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 06:42:43,429 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 06:42:43,429 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 06:42:43,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.151.212:37931/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 06:42:43,735 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 06:42:44.290942 8799 mini_ranger_kms.cc:310] Created kms service
I20260928 06:42:44.499840 8799 mini_ranger_kms.cc:326] Created kudu user
I20260928 06:42:44.542347 8799 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 06:42:44 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577761, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:44.959679 8799 mini_ranger_kms.cc:384] Added ranger policy
I20260928 06:42:44.959843 8799 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYzMBXE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYzMBXE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 06:42:44.959864 8799 mini_ranger_kms.cc:240] Using host: 127.8.151.212
I20260928 06:42:44.963634 8799 mini_ranger_kms.cc:276] Ranger KMS PID: 20982
I20260928 06:42:44.963704 8799 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.151.212:37255
Sep 28, 2026 6:42:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 6:42:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 6:42:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 6:42:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 6:42:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 6:42:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 06:42:47.076154 8799 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.176s user 0.012s sys 0.155s
I20260928 06:42:47.076279 8799 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 06:42:47.076356 8799 mini_ranger_kms.cc:401] 127.8.151.212:37255/kms/v1/keys
Sep 28 06:42:47 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790577767, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.151.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 06:42:47 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577767, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.151.212@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
Sep 28 06:42:48 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577767, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.151.212@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
Sep 28 06:42:50 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:42:50.882886 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:40765
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:40765
--ranger_config_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:37255/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:42:51.045332 21032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:51.045689 21032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:51.045744 21032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:51.051240 21032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:42:51.051321 21032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:51.051347 21032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:51.051366 21032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:42:51.051422 21032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:42:51.059226 21032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.151.212:37255/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:40765
--ranger_config_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:40765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:51.060624 21032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:51.061908 21032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:51.072934 21038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:51.072983 21040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:51.073166 21037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:51.073398 21032 server_base.cc:1034] running on GCE node
I20260928 06:42:51.073987 21032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:51.075114 21032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:51.076308 21032 hybrid_clock.cc:648] HybridClock initialized: now 1790577771076284 us; error 36 us; skew 500 ppm
Sep 28 06:42:51 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577771, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:51.082062 21032 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:42:51.083014 21032 webserver.cc:460] Webserver started at http://127.8.151.254:38409/ using document root <none> and password file <none>
I20260928 06:42:51.083575 21032 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:51.083632 21032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:42:51.083864 21032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 06:42:51 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577771, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:51.139240 21032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "3b4fd024314b4f22bfe45ac20e655393"
format_stamp: "Formatted at 2026-09-28 06:42:51 on dist-test-slave-gtr8"
server_key: "376a103b270394d83d54db8ee9897745"
server_key_iv: "219d4b2fb232dda4a21adadf534934c6"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:51.139909 21032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "3b4fd024314b4f22bfe45ac20e655393"
format_stamp: "Formatted at 2026-09-28 06:42:51 on dist-test-slave-gtr8"
server_key: "376a103b270394d83d54db8ee9897745"
server_key_iv: "219d4b2fb232dda4a21adadf534934c6"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:51.143826 21032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260928 06:42:51.161028 21047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:51.162441 21032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 06:42:51.162569 21032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "3b4fd024314b4f22bfe45ac20e655393"
format_stamp: "Formatted at 2026-09-28 06:42:51 on dist-test-slave-gtr8"
server_key: "376a103b270394d83d54db8ee9897745"
server_key_iv: "219d4b2fb232dda4a21adadf534934c6"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:51.162698 21032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:51.180836 21032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:51.183770 21032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:51.183981 21032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:51.191922 21032 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:40765
I20260928 06:42:51.191956 21099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:40765 every 8 connection(s)
I20260928 06:42:51.193028 21032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:42:51.195772 21100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:42:51.201434 21100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: Bootstrap starting.
I20260928 06:42:51.201990 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 21032
I20260928 06:42:51.202102 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:42:51.202396 8799 external_mini_cluster.cc:1428] Setting key 1d403a110d29bef2177ef1a4c3a35d6f
I20260928 06:42:51.204958 21100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: Neither blocks nor log segments found. Creating new log.
I20260928 06:42:51.205986 21100 log.cc:824] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:51.208521 21100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: No bootstrap required, opened a new log
I20260928 06:42:51.213201 21100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } }
I20260928 06:42:51.213595 21100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Sep 28 06:42:51 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:51.213725 21100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b4fd024314b4f22bfe45ac20e655393, State: Initialized, Role: FOLLOWER
I20260928 06:42:51.214238 21100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } }
I20260928 06:42:51.214478 21100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:42:51.214581 21100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:42:51.214717 21100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:42:51.215903 21100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } }
I20260928 06:42:51.216327 21100 leader_election.cc:302] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b4fd024314b4f22bfe45ac20e655393; no voters:
I20260928 06:42:51.216784 21100 leader_election.cc:288] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:42:51.218202 21100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:42:51.219952 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:51.203834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43008 (local address 127.8.151.254:40765)
0928 06:42:51.204295 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:51.204316 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:51.204361 (+ 45us) server_negotiation.cc:407] Connection header received
0928 06:42:51.205209 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:51.205233 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:51.205701 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:51.207114 (+ 1413us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:51.207578 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:51.209045 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:51.210318 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:51.211040 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:51.211642 (+ 602us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:51.214435 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:51.214441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:51.214459 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:51.214491 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:51.216356 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:51.216840 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:51.216846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:51.216852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:51.216972 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:51.217273 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:51.217275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:51.217277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:51.217753 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:51.218577 (+ 824us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:51.218721 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 06:42:51.218996 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":125,"threads_started":1}
I20260928 06:42:51.220423 21100 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 06:42:51.220824 21105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:42:51.221066 21105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 1 LEADER]: Becoming Leader. State: Replica: 3b4fd024314b4f22bfe45ac20e655393, State: Running, Role: LEADER
I20260928 06:42:51.221613 21105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } }
I20260928 06:42:51.223299 21105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b4fd024314b4f22bfe45ac20e655393. Latest consensus state: current_term: 1 leader_uuid: "3b4fd024314b4f22bfe45ac20e655393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } } }
I20260928 06:42:51.224035 21105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [sys.catalog]: This master's current role is: LEADER
I20260928 06:42:51.223924 21106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b4fd024314b4f22bfe45ac20e655393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } } }
I20260928 06:42:51.224334 21106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [sys.catalog]: This master's current role is: LEADER
I20260928 06:42:51.224603 21120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:42:51.231168 21120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:42:51.251222 21120 catalog_manager.cc:1347] Generated new cluster ID: b268aeed685147a6ada4b7bbac6604db
I20260928 06:42:51.251312 21120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:42:51.273993 21120 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:42:51.275342 21120 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:42:51.285316 21120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: Generated new TSK 0
I20260928 06:42:51.286309 21120 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:42:52.095183 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:37255/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40765
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:42:52.268105 21145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:52.268373 21145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:52.268432 21145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:52.272771 21145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:52.272832 21145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:52.272908 21145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:42:52.278432 21145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.151.212:37255/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:52.279582 21145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:52.280633 21145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:52.291505 21151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:52.291572 21153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:52.291512 21150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:52.291718 21145 server_base.cc:1034] running on GCE node
I20260928 06:42:52.292207 21145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:52.292829 21145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:52.293998 21145 hybrid_clock.cc:648] HybridClock initialized: now 1790577772293969 us; error 42 us; skew 500 ppm
Sep 28 06:42:52 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:52.296850 21145 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:42:52.297849 21145 webserver.cc:460] Webserver started at http://127.8.151.193:45243/ using document root <none> and password file <none>
I20260928 06:42:52.298475 21145 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:52.298552 21145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:42:52.298774 21145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 06:42:52 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:52.314308 21145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "6482024c978949b69db9d0ede4851189"
format_stamp: "Formatted at 2026-09-28 06:42:52 on dist-test-slave-gtr8"
server_key: "1a273cc6bba13548465ca44e1430ec43"
server_key_iv: "fb04222653f31717df65d72e94e6742a"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:52.315095 21145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "6482024c978949b69db9d0ede4851189"
format_stamp: "Formatted at 2026-09-28 06:42:52 on dist-test-slave-gtr8"
server_key: "1a273cc6bba13548465ca44e1430ec43"
server_key_iv: "fb04222653f31717df65d72e94e6742a"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:52.320168 21145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260928 06:42:52.336696 21160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:52.338526 21145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:42:52.338713 21145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "6482024c978949b69db9d0ede4851189"
format_stamp: "Formatted at 2026-09-28 06:42:52 on dist-test-slave-gtr8"
server_key: "1a273cc6bba13548465ca44e1430ec43"
server_key_iv: "fb04222653f31717df65d72e94e6742a"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:52.338897 21145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:52.356822 21145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:52.372205 21145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:52.372560 21145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:52.373530 21145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:42:52.374938 21145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:42:52.375053 21145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260928 06:42:52.375208 21145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:42:52.375299 21145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:52.386991 21145 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:42529
I20260928 06:42:52.388393 21145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:42:52.392057 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 21145
I20260928 06:42:52.392179 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:42:52.392479 8799 external_mini_cluster.cc:1428] Setting key 300d16ec918b1f626c768e643e1ac669
I20260928 06:42:52.392608 21275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:42529 every 8 connection(s)
Sep 28 06:42:52 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:52.416848 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:52.397504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:41313 (local address 127.8.151.254:40765)
0928 06:42:52.401266 (+ 3762us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:52.401270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:52.401285 (+ 15us) server_negotiation.cc:407] Connection header received
0928 06:42:52.401344 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:52.401347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:52.401397 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:52.401482 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:52.403225 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:52.404027 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:52.405916 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:52.406268 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:52.406349 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:52.412461 (+ 6112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:52.412465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:52.412467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:52.412497 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:52.414224 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:52.414860 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:52.414864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:52.414866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:52.414959 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:52.415419 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:52.415422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:52.415424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:52.415646 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:52.415726 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:52.416530 (+ 804us) server_negotiation.cc:299] Negotiation successful
0928 06:42:52.416663 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3678,"thread_start_us":64,"threads_started":1}
I20260928 06:42:52.418869 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:52.396341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:40765 (local address 127.8.151.193:41313)
0928 06:42:52.399542 (+ 3201us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:52.399606 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:52.400822 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:52.401801 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:52.401811 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:52.402269 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:52.403014 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:52.403041 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:52.404187 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:52.404191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:52.405689 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:52.405707 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:52.406573 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:52.406576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:52.406801 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:52.408962 (+ 2161us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:52.408984 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:52.412225 (+ 3241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:52.414360 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:52.414367 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:52.414385 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:52.414732 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:52.415076 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:52.415079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:52.415080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:52.415314 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:52.415790 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:52.415797 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:52.416241 (+ 444us) client_negotiation.cc:769] Sending connection context
0928 06:42:52.417319 (+ 1078us) client_negotiation.cc:240] Negotiation successful
0928 06:42:52.417678 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2916,"thread_start_us":189,"threads_started":1}
I20260928 06:42:52.420610 21276 heartbeater.cc:347] Connected to a master server at 127.8.151.254:40765
I20260928 06:42:52.420843 21276 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:52.421440 21276 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
I20260928 06:42:52.423221 21061 ts_manager.cc:194] Registered new tserver with Master: 6482024c978949b69db9d0ede4851189 (127.8.151.193:42529)
I20260928 06:42:52.424651 21061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:41313
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
Sep 28 06:42:52 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790577772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
W20260928 06:42:52.640129 8799 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.151.194: real 0.246s user 0.000s sys 0.004s
I20260928 06:42:52.640673 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:37255/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40765
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:42:52.795032 21284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:52.795387 21284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:52.795444 21284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:52.800884 21284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:52.800961 21284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:52.801047 21284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:42:52.808131 21284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.151.212:37255/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:52.809407 21284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:52.810663 21284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:52.822885 21291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:52.823298 21293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:52.823535 21290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:52.824714 21284 server_base.cc:1034] running on GCE node
I20260928 06:42:52.825158 21284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:52.825872 21284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:52.827064 21284 hybrid_clock.cc:648] HybridClock initialized: now 1790577772827036 us; error 38 us; skew 500 ppm
Sep 28 06:42:52 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:52.830443 21284 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:42:52.831501 21284 webserver.cc:460] Webserver started at http://127.8.151.194:42247/ using document root <none> and password file <none>
I20260928 06:42:52.832163 21284 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:52.832237 21284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:42:52.832479 21284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 06:42:52 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:52.868935 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "90c3e15489374c7ba037dc312cea319c"
format_stamp: "Formatted at 2026-09-28 06:42:52 on dist-test-slave-gtr8"
server_key: "2ae0990e53f4707b122f793eac6c5488"
server_key_iv: "2b69ec51ed6354fb4fe5f949affa7ff3"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:52.869638 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "90c3e15489374c7ba037dc312cea319c"
format_stamp: "Formatted at 2026-09-28 06:42:52 on dist-test-slave-gtr8"
server_key: "2ae0990e53f4707b122f793eac6c5488"
server_key_iv: "2b69ec51ed6354fb4fe5f949affa7ff3"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:52.882962 21284 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.006s sys 0.000s
I20260928 06:42:52.925302 21300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:52.927295 21284 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.003s sys 0.000s
I20260928 06:42:52.927446 21284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "90c3e15489374c7ba037dc312cea319c"
format_stamp: "Formatted at 2026-09-28 06:42:52 on dist-test-slave-gtr8"
server_key: "2ae0990e53f4707b122f793eac6c5488"
server_key_iv: "2b69ec51ed6354fb4fe5f949affa7ff3"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:52.927605 21284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:52.962476 21284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:52.972599 21284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:52.972888 21284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:52.973845 21284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:42:52.975108 21284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:42:52.975231 21284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:52.975327 21284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:42:52.975386 21284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:52.987663 21284 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:44561
I20260928 06:42:52.989091 21284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:42:52.992089 21413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:44561 every 8 connection(s)
I20260928 06:42:52.993340 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 21284
I20260928 06:42:52.993453 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:42:52.993739 8799 external_mini_cluster.cc:1428] Setting key 00cab32479de5a513805531486467ea2
Sep 28 06:42:53 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
I20260928 06:42:53.016037 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:52.993104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:40765 (local address 127.8.151.194:53231)
0928 06:42:52.993620 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:52.993674 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:52.995019 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:52.995824 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:52.995833 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:52.996295 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:52.997096 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:52.997119 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:52.998160 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:52.998163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:52.999655 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:52.999666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:53.000371 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:53.000375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:53.000633 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:53.002774 (+ 2141us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:53.002796 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:53.005211 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:53.012373 (+ 7162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:53.012382 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:53.012404 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:53.012911 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:53.013241 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:53.013244 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:53.013247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:53.013442 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:53.013937 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:53.013944 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:53.014298 (+ 354us) client_negotiation.cc:769] Sending connection context
0928 06:42:53.014517 (+ 219us) client_negotiation.cc:240] Negotiation successful
0928 06:42:53.014880 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":155,"threads_started":1}
I20260928 06:42:53.016811 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:52.992657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:53231 (local address 127.8.151.254:40765)
0928 06:42:52.992926 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:52.992931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:52.993794 (+ 863us) server_negotiation.cc:407] Connection header received
0928 06:42:52.995287 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:52.995290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:52.995343 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:52.995401 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:52.997262 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:52.998008 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:52.999787 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:53.000094 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:53.000169 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:53.005454 (+ 5285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:53.005458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:53.005461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:53.005492 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:53.012197 (+ 6705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:53.013036 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:53.013040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:53.013042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:53.013140 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:53.013558 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:53.013561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:53.013563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:53.013795 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:53.013881 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:53.016527 (+ 2646us) server_negotiation.cc:299] Negotiation successful
0928 06:42:53.016663 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
I20260928 06:42:53.017764 21414 heartbeater.cc:347] Connected to a master server at 127.8.151.254:40765
I20260928 06:42:53.018527 21414 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:53.019157 21414 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
I20260928 06:42:53.020256 21061 ts_manager.cc:194] Registered new tserver with Master: 90c3e15489374c7ba037dc312cea319c (127.8.151.194:44561)
I20260928 06:42:53.020968 21061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:53231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:42:53.037803 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:37255/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40765
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:42:53.190498 21420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:53.190892 21420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:53.191021 21420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:53.202874 21420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:53.203042 21420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:53.203186 21420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:42:53.210891 21420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.151.212:37255/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:40765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:53.212713 21420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:53.214119 21420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:53.232440 21425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:53.232430 21428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:53.233160 21426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Sep 28 06:42:53 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:53.234655 21420 server_base.cc:1034] running on GCE node
I20260928 06:42:53.235252 21420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:53.236076 21420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:53.237241 21420 hybrid_clock.cc:648] HybridClock initialized: now 1790577773237177 us; error 75 us; skew 500 ppm
Sep 28 06:42:53 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:53.241077 21420 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:42:53.242208 21420 webserver.cc:460] Webserver started at http://127.8.151.195:32929/ using document root <none> and password file <none>
I20260928 06:42:53.242920 21420 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:53.243053 21420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:42:53.243336 21420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 06:42:53 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:53.271761 21420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "8dcb023c96a04242965a41e48a2526c5"
format_stamp: "Formatted at 2026-09-28 06:42:53 on dist-test-slave-gtr8"
server_key: "6714d0d2919b3c7bdca3847194844bc1"
server_key_iv: "41ce0bcb7141049c4d85d5f8d665c9cb"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:53.276472 21420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "8dcb023c96a04242965a41e48a2526c5"
format_stamp: "Formatted at 2026-09-28 06:42:53 on dist-test-slave-gtr8"
server_key: "6714d0d2919b3c7bdca3847194844bc1"
server_key_iv: "41ce0bcb7141049c4d85d5f8d665c9cb"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:53.287019 21420 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260928 06:42:53.308640 21435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:53.316423 21420 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260928 06:42:53.316581 21420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "8dcb023c96a04242965a41e48a2526c5"
format_stamp: "Formatted at 2026-09-28 06:42:53 on dist-test-slave-gtr8"
server_key: "6714d0d2919b3c7bdca3847194844bc1"
server_key_iv: "41ce0bcb7141049c4d85d5f8d665c9cb"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:53.316720 21420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:53.353374 21420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:53.356986 21420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:53.359588 21420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:53.360422 21420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:42:53.361476 21420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:42:53.361577 21420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:53.361665 21420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:42:53.365795 21420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.000s
I20260928 06:42:53.391331 21420 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:41597
I20260928 06:42:53.392520 21420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:42:53.393474 21548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:41597 every 8 connection(s)
I20260928 06:42:53.400135 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 21420
I20260928 06:42:53.400257 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:42:53.400540 8799 external_mini_cluster.cc:1428] Setting key 4d3efaf8bbb11651f689ae5bbeae61eb
Sep 28 06:42:53 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:53.416940 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:53.403344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:60227 (local address 127.8.151.254:40765)
0928 06:42:53.403472 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:53.403477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:53.404759 (+ 1282us) server_negotiation.cc:407] Connection header received
0928 06:42:53.405977 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:53.405993 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:53.406045 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:53.406133 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:53.407505 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:53.408320 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:53.409883 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:53.410192 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:53.410261 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:53.413134 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:53.413137 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:53.413140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:53.413165 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:53.414660 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:53.415252 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:53.415256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:53.415258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:53.415342 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:53.415731 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:53.415734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:53.415735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:53.415936 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:53.416014 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:53.416653 (+ 639us) server_negotiation.cc:299] Negotiation successful
0928 06:42:53.416781 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:42:53.418641 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:53.403937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:40765 (local address 127.8.151.195:60227)
0928 06:42:53.404600 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:53.404663 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:53.405757 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:53.406311 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:53.406319 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:53.406690 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:53.407357 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:53.407377 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:53.408460 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:53.408463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:53.409754 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:53.409765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:53.410459 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:53.410463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:53.410651 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:53.411304 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:53.411324 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:53.412954 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:53.414782 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:53.414788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:53.414803 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:53.415111 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:53.415460 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:53.415462 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:53.415465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:53.415633 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:53.416077 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:53.416083 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:53.416384 (+ 301us) client_negotiation.cc:769] Sending connection context
0928 06:42:53.417156 (+ 772us) client_negotiation.cc:240] Negotiation successful
0928 06:42:53.417498 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":159,"threads_started":1}
I20260928 06:42:53.423231 21549 heartbeater.cc:347] Connected to a master server at 127.8.151.254:40765
I20260928 06:42:53.423497 21549 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:53.424081 21549 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
I20260928 06:42:53.425133 21061 ts_manager.cc:194] Registered new tserver with Master: 8dcb023c96a04242965a41e48a2526c5 (127.8.151.195:41597)
I20260928 06:42:53.425813 21061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:60227
I20260928 06:42:53.428104 21276 heartbeater.cc:503] Master 127.8.151.254:40765 was elected leader, sending a full tablet report...
I20260928 06:42:53.432174 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:42:53.432855 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 21145
I20260928 06:42:53.440140 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 21284
I20260928 06:42:53.445624 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 21420
I20260928 06:42:53.452390 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 21032
I20260928 06:42:53.459375 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:40765
--webserver_interface=127.8.151.254
--webserver_port=38409
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:40765
--ranger_config_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:37255/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.151.212:37255/kms with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:42:53.570912 21552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:53.571215 21552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:53.571283 21552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:53.575331 21552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:42:53.575393 21552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:53.575419 21552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:53.575450 21552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:42:53.575472 21552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:42:53.580899 21552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.151.212:37255/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:40765
--ranger_config_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:40765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=38409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:53.581948 21552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:53.582937 21552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:53.591399 21558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:53.591373 21557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:53.591673 21552 server_base.cc:1034] running on GCE node
W20260928 06:42:53.591409 21560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:53.592293 21552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:53.593385 21552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:53.594555 21552 hybrid_clock.cc:648] HybridClock initialized: now 1790577773594532 us; error 37 us; skew 500 ppm
Sep 28 06:42:53 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:53.597204 21552 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:42:53.598107 21552 webserver.cc:460] Webserver started at http://127.8.151.254:38409/ using document root <none> and password file <none>
I20260928 06:42:53.598613 21552 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:53.598681 21552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 06:42:53 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:53.656527 21552 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260928 06:42:53.658849 21568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:53.660233 21552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:42:53.660367 21552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "3b4fd024314b4f22bfe45ac20e655393"
format_stamp: "Formatted at 2026-09-28 06:42:51 on dist-test-slave-gtr8"
server_key: "376a103b270394d83d54db8ee9897745"
server_key_iv: "219d4b2fb232dda4a21adadf534934c6"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:53.660969 21552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:53.675765 21552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:53.678620 21552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:53.678826 21552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:53.687822 21552 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:40765
I20260928 06:42:53.687842 21620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:40765 every 8 connection(s)
I20260928 06:42:53.688942 21552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:42:53.693749 21621 sys_catalog.cc:263] Verifying existing consensus state
I20260928 06:42:53.695425 21621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: Bootstrap starting.
I20260928 06:42:53.696664 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 21552
I20260928 06:42:53.697129 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:37255/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:42529
--local_ip_for_outbound_sockets=127.8.151.193
--tserver_master_addrs=127.8.151.254:40765
--webserver_port=45243
--webserver_interface=127.8.151.193
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.151.212:37255/kms with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:42:53.717002 21621 log.cc:824] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:53.725385 21621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:42:53.725834 21621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: Bootstrap complete.
I20260928 06:42:53.731272 21621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } }
I20260928 06:42:53.731765 21621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b4fd024314b4f22bfe45ac20e655393, State: Initialized, Role: FOLLOWER
I20260928 06:42:53.732295 21621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } }
I20260928 06:42:53.732442 21621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:42:53.732506 21621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:42:53.732594 21621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 1 FOLLOWER]: Advancing to term 2
I20260928 06:42:53.734910 21621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } }
I20260928 06:42:53.735281 21621 leader_election.cc:302] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b4fd024314b4f22bfe45ac20e655393; no voters:
I20260928 06:42:53.735625 21621 leader_election.cc:288] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 06:42:53.735824 21625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 06:42:53.736114 21625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [term 2 LEADER]: Becoming Leader. State: Replica: 3b4fd024314b4f22bfe45ac20e655393, State: Running, Role: LEADER
I20260928 06:42:53.736605 21625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } }
I20260928 06:42:53.736856 21621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:42:53.737649 21625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b4fd024314b4f22bfe45ac20e655393. Latest consensus state: current_term: 2 leader_uuid: "3b4fd024314b4f22bfe45ac20e655393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } } }
I20260928 06:42:53.737839 21625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [sys.catalog]: This master's current role is: LEADER
I20260928 06:42:53.738004 21627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3b4fd024314b4f22bfe45ac20e655393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4fd024314b4f22bfe45ac20e655393" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40765 } } }
I20260928 06:42:53.738091 21627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393 [sys.catalog]: This master's current role is: LEADER
I20260928 06:42:53.739054 21621 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 06:42:53.739758 21634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:42:53.745935 21634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:42:53.748859 21634 catalog_manager.cc:1259] Loaded cluster ID: b268aeed685147a6ada4b7bbac6604db
I20260928 06:42:53.751956 21634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:42:53.754458 21634 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:42:53.756980 21634 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: Loaded TSK: 0
I20260928 06:42:53.758248 21634 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 06:42:53.857547 21623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:53.857882 21623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:53.857944 21623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:53.863966 21623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:53.864053 21623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:53.864142 21623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:42:53.871834 21623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.151.212:37255/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:42529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=45243
--tserver_master_addrs=127.8.151.254:40765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:53.873153 21623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:53.874482 21623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:53.887818 21658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:53.888130 21623 server_base.cc:1034] running on GCE node
W20260928 06:42:53.888284 21659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:53.888687 21661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:53.890368 21623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:53.891249 21623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:53.892443 21623 hybrid_clock.cc:648] HybridClock initialized: now 1790577773892408 us; error 49 us; skew 500 ppm
Sep 28 06:42:53 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:53.896049 21623 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:42:53.897194 21623 webserver.cc:460] Webserver started at http://127.8.151.193:45243/ using document root <none> and password file <none>
I20260928 06:42:53.897886 21623 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:53.897977 21623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 06:42:53 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:53.923676 21623 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 06:42:53.926903 21669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:53.928615 21623 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260928 06:42:53.928741 21623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "6482024c978949b69db9d0ede4851189"
format_stamp: "Formatted at 2026-09-28 06:42:52 on dist-test-slave-gtr8"
server_key: "1a273cc6bba13548465ca44e1430ec43"
server_key_iv: "fb04222653f31717df65d72e94e6742a"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:53.929458 21623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:53.960326 21623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:53.963333 21623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:53.963515 21623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:53.964090 21623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:42:53.964968 21623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:42:53.965015 21623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:53.965083 21623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:42:53.965104 21623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:53.976617 21623 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:42529
I20260928 06:42:53.977919 21623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:42:53.978833 21782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:42529 every 8 connection(s)
I20260928 06:42:53.985457 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 21623
I20260928 06:42:53.986281 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:37255/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:44561
--local_ip_for_outbound_sockets=127.8.151.194
--tserver_master_addrs=127.8.151.254:40765
--webserver_port=42247
--webserver_interface=127.8.151.194
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.151.212:37255/kms with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 06:42:53 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:54.005831 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:53.982303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:40765 (local address 127.8.151.193:51803)
0928 06:42:53.984697 (+ 2394us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:53.984755 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:53.985799 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:53.988249 (+ 2450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:53.988266 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:53.988629 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:53.989259 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:53.989279 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:53.991397 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:53.991401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:53.992718 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:53.992729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:53.993933 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:53.993937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:53.994117 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:53.997618 (+ 3501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:53.997643 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:53.999370 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:54.001953 (+ 2583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:54.001960 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:54.001976 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:54.002312 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:54.002793 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:54.002796 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:54.002797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:54.002973 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:54.003830 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:54.003836 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:54.004156 (+ 320us) client_negotiation.cc:769] Sending connection context
0928 06:42:54.004363 (+ 207us) client_negotiation.cc:240] Negotiation successful
0928 06:42:54.004728 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2159,"thread_start_us":141,"threads_started":1}
I20260928 06:42:54.008397 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:53.983928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:51803 (local address 127.8.151.254:40765)
0928 06:42:53.986329 (+ 2401us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:53.986351 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:53.986398 (+ 47us) server_negotiation.cc:407] Connection header received
0928 06:42:53.986716 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:53.986732 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:53.987177 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:53.987548 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:53.989567 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:53.991242 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:53.992857 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:53.993554 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:53.993707 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:53.999667 (+ 5960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:53.999674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:53.999696 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:53.999733 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:54.001807 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:54.002511 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:54.002519 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:54.002524 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:54.002673 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:54.003077 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:54.003080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:54.003081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:54.003634 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:54.003774 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:54.006385 (+ 2611us) server_negotiation.cc:299] Negotiation successful
0928 06:42:54.006697 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2230,"thread_start_us":116,"threads_started":1}
I20260928 06:42:54.020853 21783 heartbeater.cc:347] Connected to a master server at 127.8.151.254:40765
I20260928 06:42:54.021126 21783 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:54.021750 21783 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
W20260928 06:42:54.024220 21783 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:40765 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 06:42:54.024344 21783 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:54.024531 21783 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
I20260928 06:42:54.025648 21783 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:54.025851 21783 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
W20260928 06:42:54.026741 21783 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 06:42:54.164741 21787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:54.165127 21787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:54.165275 21787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:54.170280 21787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:54.170404 21787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:54.170526 21787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:42:54.176923 21787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.151.212:37255/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:44561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=42247
--tserver_master_addrs=127.8.151.254:40765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:54.178148 21787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:54.179442 21787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:54.192236 21795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:54.192700 21793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:54.193691 21792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:54.196606 21787 server_base.cc:1034] running on GCE node
I20260928 06:42:54.197340 21787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:54.198256 21787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:54.199445 21787 hybrid_clock.cc:648] HybridClock initialized: now 1790577774199409 us; error 53 us; skew 500 ppm
Sep 28 06:42:54 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577774, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:54.203132 21787 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:42:54.204361 21787 webserver.cc:460] Webserver started at http://127.8.151.194:42247/ using document root <none> and password file <none>
I20260928 06:42:54.205286 21787 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:54.205407 21787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 06:42:54 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577774, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:54.227515 21787 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260928 06:42:54.233059 21803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:54.234804 21787 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260928 06:42:54.235080 21787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "90c3e15489374c7ba037dc312cea319c"
format_stamp: "Formatted at 2026-09-28 06:42:52 on dist-test-slave-gtr8"
server_key: "2ae0990e53f4707b122f793eac6c5488"
server_key_iv: "2b69ec51ed6354fb4fe5f949affa7ff3"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:54.235958 21787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:54.257251 21787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:54.260707 21787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:54.260897 21787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:54.261644 21787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:42:54.262720 21787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:42:54.262784 21787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:54.262843 21787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:42:54.262872 21787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:54.278175 21787 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:44561
I20260928 06:42:54.282171 21787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:42:54.286820 21916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:44561 every 8 connection(s)
I20260928 06:42:54.293664 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 21787
I20260928 06:42:54.294142 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.151.212:37255/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:41597
--local_ip_for_outbound_sockets=127.8.151.195
--tserver_master_addrs=127.8.151.254:40765
--webserver_port=32929
--webserver_interface=127.8.151.195
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.151.212:37255/kms with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 06:42:54 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577774, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:54.303257 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:54.287559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:56501 (local address 127.8.151.254:40765)
0928 06:42:54.287700 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:54.287705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:54.289130 (+ 1425us) server_negotiation.cc:407] Connection header received
0928 06:42:54.290803 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:54.290807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:54.290871 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:54.290941 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:54.292998 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:54.293742 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:54.295403 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:54.295736 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:54.295813 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:54.298756 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:54.298760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:54.298763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:54.298795 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:54.300582 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:54.301286 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:54.301290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:54.301292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:54.301384 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:54.301812 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:54.301816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:54.301817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:54.302068 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:54.302145 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:54.302895 (+ 750us) server_negotiation.cc:299] Negotiation successful
0928 06:42:54.303065 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 06:42:54.305158 21919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:54.288132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:40765 (local address 127.8.151.194:56501)
0928 06:42:54.288645 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:54.289012 (+ 367us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:54.290570 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:54.291280 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:54.291290 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:54.291817 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:54.292783 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:54.292806 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:54.293912 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:54.293916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:54.295265 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:54.295276 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:54.296051 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:54.296055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:54.296253 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:54.296906 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:54.296928 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:54.298551 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:54.300719 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:54.300725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:54.300743 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:54.301163 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:54.301498 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:54.301501 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:54.301503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:54.301696 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:54.302233 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:54.302241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:54.302587 (+ 346us) client_negotiation.cc:769] Sending connection context
0928 06:42:54.303661 (+ 1074us) client_negotiation.cc:240] Negotiation successful
0928 06:42:54.304016 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":132,"threads_started":1}
I20260928 06:42:54.309297 21917 heartbeater.cc:347] Connected to a master server at 127.8.151.254:40765
I20260928 06:42:54.309551 21917 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:54.310141 21917 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
W20260928 06:42:54.311947 21917 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:40765 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 06:42:54.312074 21917 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:54.312237 21917 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
I20260928 06:42:54.313081 21917 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:54.313234 21917 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
W20260928 06:42:54.313948 21917 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 06:42:54.470129 21920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:54.470542 21920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:54.470638 21920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:54.476519 21920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:54.476598 21920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:54.476696 21920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:42:54.484537 21920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.151.212:37255/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:41597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=32929
--tserver_master_addrs=127.8.151.254:40765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:54.486006 21920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:54.487428 21920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:54.499780 21926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:54.499801 21927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:54.500023 21920 server_base.cc:1034] running on GCE node
W20260928 06:42:54.499773 21929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:54.500658 21920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:54.501364 21920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:54.502548 21920 hybrid_clock.cc:648] HybridClock initialized: now 1790577774502519 us; error 47 us; skew 500 ppm
Sep 28 06:42:54 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577774, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:54.505874 21920 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:42:54.506852 21920 webserver.cc:460] Webserver started at http://127.8.151.195:32929/ using document root <none> and password file <none>
I20260928 06:42:54.507560 21920 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:54.507648 21920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 06:42:54 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577774, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:54.535073 21920 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260928 06:42:54.538182 21937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:54.539721 21920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:42:54.539897 21920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "8dcb023c96a04242965a41e48a2526c5"
format_stamp: "Formatted at 2026-09-28 06:42:53 on dist-test-slave-gtr8"
server_key: "6714d0d2919b3c7bdca3847194844bc1"
server_key_iv: "41ce0bcb7141049c4d85d5f8d665c9cb"
server_key_version: "kuduclusterkey@0"
I20260928 06:42:54.540686 21920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:54.573218 21920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:54.576882 21920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:54.577153 21920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:54.577905 21920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:42:54.578951 21920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:42:54.579066 21920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:54.579218 21920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:42:54.579282 21920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:54.590030 21920 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:41597
I20260928 06:42:54.592099 21920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:42:54.593358 22050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:41597 every 8 connection(s)
I20260928 06:42:54.600574 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 21920
Sep 28 06:42:54 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577774, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:54.610121 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:54.594924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:40765 (local address 127.8.151.195:44127)
0928 06:42:54.595497 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:54.595558 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:54.596624 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:54.597266 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:54.597275 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:54.597626 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:54.598321 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:54.598341 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:54.599455 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:54.599458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:54.600825 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:54.600836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:54.601551 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:54.601555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:54.601744 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:54.602385 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:54.602405 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:54.604104 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:54.606253 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:54.606259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:54.606276 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:54.606643 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:54.607018 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:54.607022 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:54.607024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:54.607500 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:54.608066 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:54.608072 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:54.608419 (+ 347us) client_negotiation.cc:769] Sending connection context
0928 06:42:54.608641 (+ 222us) client_negotiation.cc:240] Negotiation successful
0928 06:42:54.609001 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":182,"threads_started":1}
I20260928 06:42:54.611032 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:54.594391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:44127 (local address 127.8.151.254:40765)
0928 06:42:54.594524 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:54.594528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:54.595665 (+ 1137us) server_negotiation.cc:407] Connection header received
0928 06:42:54.596896 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:54.596900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:54.596963 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:54.597070 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:54.598495 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:54.599302 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:54.600956 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:54.601277 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:54.601352 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:54.604304 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:54.604308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:54.604311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:54.604343 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:54.606101 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:54.606778 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:54.606782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:54.606784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:54.606901 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:54.607628 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:54.607632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:54.607634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:54.607895 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:54.608005 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:54.610708 (+ 2703us) server_negotiation.cc:299] Negotiation successful
0928 06:42:54.610866 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 06:42:54.612238 22051 heartbeater.cc:347] Connected to a master server at 127.8.151.254:40765
I20260928 06:42:54.612495 22051 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:54.613250 22051 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
W20260928 06:42:54.615031 22051 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:40765 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 06:42:54.615180 22051 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:54.615357 22051 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
I20260928 06:42:54.616374 22051 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:54.616426 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:54.604892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.8.151.254:40765)
0928 06:42:54.605821 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:54.605826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:54.605838 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:42:54.605870 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:54.605873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:54.605916 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:54.605984 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:54.606900 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:54.607857 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:54.609090 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:54.609408 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:54.609496 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:54.612086 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:54.612090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:54.612093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:54.612116 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:54.613873 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:54.614631 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:54.614634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:54.614635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:54.614720 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:54.615022 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:54.615025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:54.615027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:54.615273 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:54.615479 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:54.616126 (+ 647us) server_negotiation.cc:299] Negotiation successful
0928 06:42:54.616253 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":140,"threads_started":1}
I20260928 06:42:54.616546 22051 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
W20260928 06:42:54.617424 22051 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 06:42:55.026983 21783 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:55.027333 21783 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
I20260928 06:42:55.028553 21585 ts_manager.cc:194] Registered new tserver with Master: 6482024c978949b69db9d0ede4851189 (127.8.151.193:42529)
I20260928 06:42:55.030228 21585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:51803
Sep 28 06:42:55 dist-test-slave-gtr8 krb5kdc[20587](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790577775, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:55.314170 21917 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:55.314491 21917 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
I20260928 06:42:55.315402 21585 ts_manager.cc:194] Registered new tserver with Master: 90c3e15489374c7ba037dc312cea319c (127.8.151.194:44561)
I20260928 06:42:55.316200 21585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:56501
Sep 28 06:42:55 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577775, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
I20260928 06:42:55.617660 22051 heartbeater.cc:464] Registering TS with master...
I20260928 06:42:55.617945 22051 heartbeater.cc:511] Master 127.8.151.254:40765 requested a full tablet report, sending...
I20260928 06:42:55.618726 21585 ts_manager.cc:194] Registered new tserver with Master: 8dcb023c96a04242965a41e48a2526c5 (127.8.151.195:41597)
I20260928 06:42:55.619495 21585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:44127
I20260928 06:42:55.623126 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:42:55.632582 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.624673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.8.151.254:40765)
0928 06:42:55.624960 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:55.624964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:55.625290 (+ 326us) server_negotiation.cc:407] Connection header received
0928 06:42:55.625385 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:55.625387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:55.625438 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:55.625520 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:55.626416 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.627259 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.628049 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.628286 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.628370 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.629636 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:55.629640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:55.629643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:55.629668 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:55.631018 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:55.631462 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:55.631465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:55.631467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:55.631551 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:55.631789 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:55.631793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:55.631794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:55.631962 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:55.632080 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:55.632301 (+ 221us) server_negotiation.cc:299] Negotiation successful
0928 06:42:55.632418 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
I20260928 06:42:55.636109 21585 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:42:55.639758 21585 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:42:55.663307 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.653691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:42529 (local address 127.0.0.1:32888)
0928 06:42:55.654197 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:55.654249 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:55.654366 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:55.657425 (+ 3059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:55.657431 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:55.657467 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:42:55.657613 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:55.657620 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.659070 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.659074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:55.661485 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:55.661491 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.662744 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.662748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:55.662865 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.662901 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 06:42:55.663005 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 06:42:55.663120 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":4,"thread_start_us":102,"threads_started":1}
I20260928 06:42:55.664158 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.654521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32888 (local address 127.8.151.193:42529)
0928 06:42:55.656912 (+ 2391us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:55.656918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:55.656943 (+ 25us) server_negotiation.cc:407] Connection header received
0928 06:42:55.657021 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:55.657027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:55.657257 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:55.657415 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:42:55.657760 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.658937 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.661598 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.662421 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.662537 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.662683 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:55.663915 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0928 06:42:55.664019 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2285,"thread_start_us":97,"threads_started":1}
I20260928 06:42:55.667387 21699 tablet_service.cc:1467] Processing CreateTablet for tablet 098f17f309ef4a8c8c7d09c706efe661 (DEFAULT_TABLE table=test-table [id=7a813c3449a44b0eae5931457a547263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:42:55.668488 21699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098f17f309ef4a8c8c7d09c706efe661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:42:55.669313 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.653837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:44561 (local address 127.0.0.1:50942)
0928 06:42:55.656580 (+ 2743us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:55.656594 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:55.656683 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:55.664949 (+ 8266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:55.664952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:55.664970 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:42:55.665086 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:55.665092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.666641 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.666645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:55.667840 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:55.667851 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.668940 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.668944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:55.669035 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.669062 (+ 27us) client_negotiation.cc:769] Sending connection context
0928 06:42:55.669111 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 06:42:55.669176 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2676,"thread_start_us":106,"threads_started":1}
I20260928 06:42:55.669926 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.654737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50942 (local address 127.8.151.194:44561)
0928 06:42:55.655076 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:55.655082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:55.664501 (+ 9419us) server_negotiation.cc:407] Connection header received
0928 06:42:55.664591 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:55.664597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:55.664801 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:55.665243 (+ 442us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:42:55.665381 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.666509 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.667993 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.668805 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.669501 (+ 696us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.669650 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:55.669700 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 06:42:55.669788 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":67,"threads_started":1}
I20260928 06:42:55.671504 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.654368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:41597 (local address 127.0.0.1:50266)
0928 06:42:55.657666 (+ 3298us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:55.657679 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:55.657757 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:55.658416 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:55.658419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:55.658438 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:42:55.658538 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:55.658544 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.660078 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.660081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:55.661331 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:55.661343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.671110 (+ 9767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.671114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:55.671230 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.671253 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:42:55.671301 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 06:42:55.671367 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3237,"mutex_wait_us":42,"thread_start_us":77,"threads_started":1}
I20260928 06:42:55.672103 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.654574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50266 (local address 127.8.151.195:41597)
0928 06:42:55.657980 (+ 3406us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:55.657986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:55.658009 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:42:55.658079 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:55.658084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:55.658271 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:55.658687 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:42:55.658817 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.659950 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.670171 (+ 10221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.670969 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.671678 (+ 709us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.671835 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:55.671886 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 06:42:55.671974 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3303,"thread_start_us":106,"threads_started":1}
I20260928 06:42:55.672825 21847 tablet_service.cc:1467] Processing CreateTablet for tablet 098f17f309ef4a8c8c7d09c706efe661 (DEFAULT_TABLE table=test-table [id=7a813c3449a44b0eae5931457a547263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:42:55.673925 21847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098f17f309ef4a8c8c7d09c706efe661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:42:55.674969 21966 tablet_service.cc:1467] Processing CreateTablet for tablet 098f17f309ef4a8c8c7d09c706efe661 (DEFAULT_TABLE table=test-table [id=7a813c3449a44b0eae5931457a547263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:42:55.675959 21966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098f17f309ef4a8c8c7d09c706efe661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:42:55.686597 22085 tablet_bootstrap.cc:492] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189: Bootstrap starting.
I20260928 06:42:55.689193 22085 tablet_bootstrap.cc:654] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189: Neither blocks nor log segments found. Creating new log.
I20260928 06:42:55.689679 22086 tablet_bootstrap.cc:492] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: Bootstrap starting.
I20260928 06:42:55.692135 22086 tablet_bootstrap.cc:654] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: Neither blocks nor log segments found. Creating new log.
I20260928 06:42:55.694162 22085 log.cc:824] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:55.695101 22086 log.cc:824] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:55.695629 22084 tablet_bootstrap.cc:492] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: Bootstrap starting.
I20260928 06:42:55.698673 22084 tablet_bootstrap.cc:654] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: Neither blocks nor log segments found. Creating new log.
I20260928 06:42:55.699609 22084 log.cc:824] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:55.704113 22085 tablet_bootstrap.cc:492] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189: No bootstrap required, opened a new log
I20260928 06:42:55.704396 22085 ts_tablet_manager.cc:1397] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260928 06:42:55.708595 22085 raft_consensus.cc:348] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 } } peers { permanent_uuid: "90c3e15489374c7ba037dc312cea319c" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44561 } } peers { permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 } }
I20260928 06:42:55.710562 22086 tablet_bootstrap.cc:492] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: No bootstrap required, opened a new log
I20260928 06:42:55.710810 22086 ts_tablet_manager.cc:1397] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.000s
I20260928 06:42:55.716228 22086 raft_consensus.cc:348] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 } } peers { permanent_uuid: "90c3e15489374c7ba037dc312cea319c" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44561 } } peers { permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 } }
I20260928 06:42:55.716634 22086 raft_consensus.cc:374] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:42:55.716750 22085 raft_consensus.cc:374] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:42:55.716873 22086 raft_consensus.cc:729] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dcb023c96a04242965a41e48a2526c5, State: Initialized, Role: FOLLOWER
I20260928 06:42:55.716964 22085 raft_consensus.cc:729] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6482024c978949b69db9d0ede4851189, State: Initialized, Role: FOLLOWER
I20260928 06:42:55.717703 22085 consensus_queue.cc:260] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 } } peers { permanent_uuid: "90c3e15489374c7ba037dc312cea319c" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44561 } } peers { permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 } }
I20260928 06:42:55.718185 22086 consensus_queue.cc:260] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 } } peers { permanent_uuid: "90c3e15489374c7ba037dc312cea319c" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44561 } } peers { permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 } }
I20260928 06:42:55.719436 22086 ts_tablet_manager.cc:1428] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260928 06:42:55.720589 22085 ts_tablet_manager.cc:1428] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260928 06:42:55.723539 22084 tablet_bootstrap.cc:492] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: No bootstrap required, opened a new log
I20260928 06:42:55.724974 22084 ts_tablet_manager.cc:1397] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.002s
I20260928 06:42:55.729177 22084 raft_consensus.cc:348] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 } } peers { permanent_uuid: "90c3e15489374c7ba037dc312cea319c" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44561 } } peers { permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 } }
I20260928 06:42:55.729928 21783 heartbeater.cc:503] Master 127.8.151.254:40765 was elected leader, sending a full tablet report...
I20260928 06:42:55.731451 22051 heartbeater.cc:503] Master 127.8.151.254:40765 was elected leader, sending a full tablet report...
W20260928 06:42:55.733196 21784 tablet.cc:2369] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:42:55.734359 22084 raft_consensus.cc:374] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:42:55.734551 22084 raft_consensus.cc:729] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c3e15489374c7ba037dc312cea319c, State: Initialized, Role: FOLLOWER
I20260928 06:42:55.735262 22084 consensus_queue.cc:260] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 } } peers { permanent_uuid: "90c3e15489374c7ba037dc312cea319c" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44561 } } peers { permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 } }
I20260928 06:42:55.736757 22084 ts_tablet_manager.cc:1428] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260928 06:42:55.739322 21917 heartbeater.cc:503] Master 127.8.151.254:40765 was elected leader, sending a full tablet report...
W20260928 06:42:55.789386 21918 tablet.cc:2369] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 06:42:55 dist-test-slave-gtr8 krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577775, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for HTTP/127.8.151.212@KRBTEST.COM
W20260928 06:42:55.848107 22052 tablet.cc:2369] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:42:55.925572 22092 raft_consensus.cc:482] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:42:55.925815 22092 raft_consensus.cc:504] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 } } peers { permanent_uuid: "90c3e15489374c7ba037dc312cea319c" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44561 } } peers { permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 } }
I20260928 06:42:55.927228 22092 leader_election.cc:288] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6482024c978949b69db9d0ede4851189 (127.8.151.193:42529), 8dcb023c96a04242965a41e48a2526c5 (127.8.151.195:41597)
I20260928 06:42:55.947517 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.928178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:35141 (local address 127.8.151.195:41597)
0928 06:42:55.928297 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:55.928301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:55.939030 (+ 10729us) server_negotiation.cc:407] Connection header received
0928 06:42:55.939253 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:55.939257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:55.939311 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:55.939378 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:42:55.941331 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.942224 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.946172 (+ 3948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.946757 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.946848 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.946902 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:55.947299 (+ 397us) server_negotiation.cc:299] Negotiation successful
0928 06:42:55.947377 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 06:42:55.948174 22095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.927671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:41597 (local address 127.8.151.194:35141)
0928 06:42:55.938892 (+ 11221us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:55.938913 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:55.939087 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:55.940973 (+ 1886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:55.940977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:55.941014 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:42:55.941173 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:55.941181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.942433 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.942438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:55.946020 (+ 3582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:55.946031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.947035 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.947039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:55.947167 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.947190 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:42:55.947931 (+ 741us) client_negotiation.cc:240] Negotiation successful
0928 06:42:55.948014 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11114,"thread_start_us":97,"threads_started":1}
I20260928 06:42:55.948951 22005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098f17f309ef4a8c8c7d09c706efe661" candidate_uuid: "90c3e15489374c7ba037dc312cea319c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dcb023c96a04242965a41e48a2526c5" is_pre_election: true
I20260928 06:42:55.949332 22005 raft_consensus.cc:2454] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90c3e15489374c7ba037dc312cea319c in term 0.
I20260928 06:42:55.949954 21805 leader_election.cc:302] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dcb023c96a04242965a41e48a2526c5, 90c3e15489374c7ba037dc312cea319c; no voters:
I20260928 06:42:55.950294 22092 raft_consensus.cc:2781] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:42:55.950366 22092 raft_consensus.cc:482] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:42:55.950402 22092 raft_consensus.cc:3037] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:42:55.951951 22092 raft_consensus.cc:504] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 } } peers { permanent_uuid: "90c3e15489374c7ba037dc312cea319c" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44561 } } peers { permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 } }
I20260928 06:42:55.952359 22092 leader_election.cc:288] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [CANDIDATE]: Term 1 election: Requested vote from peers 6482024c978949b69db9d0ede4851189 (127.8.151.193:42529), 8dcb023c96a04242965a41e48a2526c5 (127.8.151.195:41597)
I20260928 06:42:55.952662 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.928665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:56421 (local address 127.8.151.193:42529)
0928 06:42:55.928784 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:55.928788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:55.946119 (+ 17331us) server_negotiation.cc:407] Connection header received
0928 06:42:55.946306 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:55.946309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:55.946360 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:55.946408 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:42:55.946793 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.947506 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.951314 (+ 3808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.951884 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.951984 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.952035 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:55.952447 (+ 412us) server_negotiation.cc:299] Negotiation successful
0928 06:42:55.952546 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 06:42:55.952919 22005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098f17f309ef4a8c8c7d09c706efe661" candidate_uuid: "90c3e15489374c7ba037dc312cea319c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dcb023c96a04242965a41e48a2526c5"
I20260928 06:42:55.953061 22005 raft_consensus.cc:3037] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:42:55.953830 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:55.928519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:42529 (local address 127.8.151.194:56421)
0928 06:42:55.946024 (+ 17505us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:55.946040 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:55.946194 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:55.946542 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:55.946545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:55.946562 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:42:55.946672 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:55.946678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.947671 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.947674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:55.948741 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:55.948751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:55.952209 (+ 3458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:55.952213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:55.952306 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:55.952331 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 06:42:55.952919 (+ 588us) client_negotiation.cc:240] Negotiation successful
0928 06:42:55.952999 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17420,"thread_start_us":91,"threads_started":1}
I20260928 06:42:55.954341 22005 raft_consensus.cc:2454] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c3e15489374c7ba037dc312cea319c in term 1.
I20260928 06:42:55.954641 21805 leader_election.cc:302] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dcb023c96a04242965a41e48a2526c5, 90c3e15489374c7ba037dc312cea319c; no voters:
I20260928 06:42:55.954809 22092 raft_consensus.cc:2781] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:42:55.954733 21719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098f17f309ef4a8c8c7d09c706efe661" candidate_uuid: "90c3e15489374c7ba037dc312cea319c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6482024c978949b69db9d0ede4851189" is_pre_election: true
I20260928 06:42:55.955029 22092 raft_consensus.cc:686] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 1 LEADER]: Becoming Leader. State: Replica: 90c3e15489374c7ba037dc312cea319c, State: Running, Role: LEADER
I20260928 06:42:55.955063 21719 raft_consensus.cc:2454] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90c3e15489374c7ba037dc312cea319c in term 0.
I20260928 06:42:55.955385 21737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098f17f309ef4a8c8c7d09c706efe661" candidate_uuid: "90c3e15489374c7ba037dc312cea319c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6482024c978949b69db9d0ede4851189"
I20260928 06:42:55.955387 22092 consensus_queue.cc:237] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 } } peers { permanent_uuid: "90c3e15489374c7ba037dc312cea319c" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44561 } } peers { permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 } }
I20260928 06:42:55.955485 21737 raft_consensus.cc:3037] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:42:55.956732 21737 raft_consensus.cc:2454] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c3e15489374c7ba037dc312cea319c in term 1.
I20260928 06:42:55.958807 21585 catalog_manager.cc:5495] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c reported cstate change: term changed from 0 to 1, leader changed from <none> to 90c3e15489374c7ba037dc312cea319c (127.8.151.194). New cstate: current_term: 1 leader_uuid: "90c3e15489374c7ba037dc312cea319c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c3e15489374c7ba037dc312cea319c" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 44561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 } health_report { overall_health: UNKNOWN } } }
I20260928 06:42:56.035601 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:56.030323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.8.151.194:44561)
0928 06:42:56.030467 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:56.030472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:56.030498 (+ 26us) server_negotiation.cc:407] Connection header received
0928 06:42:56.030590 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:56.030594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:56.030664 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:56.030786 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:42:56.031312 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:56.032118 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:56.033361 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:56.033690 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:56.033792 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:56.034239 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:42:56.034817 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:42:56.034929 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:56.035372 (+ 443us) server_negotiation.cc:299] Negotiation successful
0928 06:42:56.035437 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 06:42:56.043403 22005 raft_consensus.cc:1261] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5 [term 1 FOLLOWER]: Refusing update from remote peer 90c3e15489374c7ba037dc312cea319c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:42:56.044282 22092 consensus_queue.cc:1035] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dcb023c96a04242965a41e48a2526c5" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 41597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:42:56.051229 21737 raft_consensus.cc:1261] T 098f17f309ef4a8c8c7d09c706efe661 P 6482024c978949b69db9d0ede4851189 [term 1 FOLLOWER]: Refusing update from remote peer 90c3e15489374c7ba037dc312cea319c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:42:56.052030 22092 consensus_queue.cc:1035] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6482024c978949b69db9d0ede4851189" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:42:56.066967 22103 mvcc.cc:205] Tried to move back new op lower bound from 7334206570664673280 to 7334206570316955648. Current Snapshot: MvccSnapshot[applied={T|T < 7334206570664673280 or (T in {7334206570664673280})}]
I20260928 06:42:56.089670 21585 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:42:56.089882 21585 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:42:56.092587 21585 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b4fd024314b4f22bfe45ac20e655393: Sending DeleteTablet for 3 replicas of tablet 098f17f309ef4a8c8c7d09c706efe661
I20260928 06:42:56.094291 21966 tablet_service.cc:1514] Processing DeleteTablet for tablet 098f17f309ef4a8c8c7d09c706efe661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:42:56 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:50266
I20260928 06:42:56.094288 21699 tablet_service.cc:1514] Processing DeleteTablet for tablet 098f17f309ef4a8c8c7d09c706efe661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:42:56 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:32888
I20260928 06:42:56.095079 21847 tablet_service.cc:1514] Processing DeleteTablet for tablet 098f17f309ef4a8c8c7d09c706efe661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:42:56 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:50942
I20260928 06:42:56.095602 22112 tablet_replica.cc:330] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: stopping tablet replica
I20260928 06:42:56.095902 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 21623
I20260928 06:42:56.095935 22112 raft_consensus.cc:2229] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:42:56.096309 22112 raft_consensus.cc:2258] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:42:56.098110 22112 ts_tablet_manager.cc:1905] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:42:56.100935 22112 ts_tablet_manager.cc:1918] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:42:56.101035 22112 log.cc:1196] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/098f17f309ef4a8c8c7d09c706efe661
I20260928 06:42:56.101344 22112 ts_tablet_manager.cc:1939] T 098f17f309ef4a8c8c7d09c706efe661 P 90c3e15489374c7ba037dc312cea319c: Deleting consensus metadata
I20260928 06:42:56.101938 22111 tablet_replica.cc:330] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: stopping tablet replica
I20260928 06:42:56.102332 22111 raft_consensus.cc:2229] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:42:56.102586 21570 catalog_manager.cc:4841] TS 90c3e15489374c7ba037dc312cea319c (127.8.151.194:44561): tablet 098f17f309ef4a8c8c7d09c706efe661 (table test-table [id=7a813c3449a44b0eae5931457a547263]) successfully deleted
I20260928 06:42:56.103199 22111 raft_consensus.cc:2258] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:42:56.108783 22111 ts_tablet_manager.cc:1905] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:42:56.111598 22111 ts_tablet_manager.cc:1918] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:42:56.115227 22111 log.cc:1196] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/098f17f309ef4a8c8c7d09c706efe661
I20260928 06:42:56.115634 22111 ts_tablet_manager.cc:1939] T 098f17f309ef4a8c8c7d09c706efe661 P 8dcb023c96a04242965a41e48a2526c5: Deleting consensus metadata
W20260928 06:42:56.116508 21572 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:42529 (error 108)
I20260928 06:42:56.117082 21570 catalog_manager.cc:4841] TS 8dcb023c96a04242965a41e48a2526c5 (127.8.151.195:41597): tablet 098f17f309ef4a8c8c7d09c706efe661 (table test-table [id=7a813c3449a44b0eae5931457a547263]) successfully deleted
I20260928 06:42:56.117660 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:56.117318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:42529 (local address 127.0.0.1:32890)
0928 06:42:56.117495 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:56.117568 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260928 06:42:56.117835 21572 catalog_manager.cc:4584] TS 6482024c978949b69db9d0ede4851189 (127.8.151.193:42529): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 098f17f309ef4a8c8c7d09c706efe661: Network error: Client connection negotiation failed: client connection to 127.8.151.193:42529: connect: Connection refused (error 111)
I20260928 06:42:56.131330 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 21787
I20260928 06:42:56.140003 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 21920
I20260928 06:42:56.148516 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 21552
I20260928 06:42:56.156335 8799 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 06:42:56.500838 8799 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 06:42:56.500985 8799 mini_ranger.cc:66] Stopping Ranger...
I20260928 06:42:56.835855 8799 mini_ranger.cc:68] Stopped Ranger
2026-09-28 06:42:56.836 UTC [20617] LOG: received smart shutdown request
2026-09-28 06:42:56.840 UTC [20617] LOG: background worker "logical replication launcher" (PID 20626) exited with exit code 1
2026-09-28 06:42:56.840 UTC [20621] LOG: shutting down
2026-09-28 06:42:56.944 UTC [20617] LOG: database system is shut down
2026-09-28T06:42:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55084 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:42:57 dist-test-slave-gtr8 krb5kdc[22123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:42:57 dist-test-slave-gtr8 krb5kdc[22123](info): set up 2 sockets
Sep 28 06:42:57 dist-test-slave-gtr8 krb5kdc[22123](info): commencing operation
krb5kdc: starting...
W20260928 06:42:57.999668 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:42:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:42:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:42:58.122010 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:43263
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:43263,127.8.151.253:45945,127.8.151.252:42111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:42:58.229492 22136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:58.229756 22136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:58.229812 22136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:58.233748 22136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:42:58.233807 22136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:58.233834 22136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:58.233857 22136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:42:58.233886 22136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:42:58.239115 22136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:43263,127.8.151.253:45945,127.8.151.252:42111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:43263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:58.240216 22136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:58.241212 22136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:58.249589 22141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:58.249648 22144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:58.249598 22142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:58.249846 22136 server_base.cc:1034] running on GCE node
I20260928 06:42:58.250620 22136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:58.252009 22136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:58.253229 22136 hybrid_clock.cc:648] HybridClock initialized: now 1790577778253188 us; error 69 us; skew 500 ppm
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:58.256418 22136 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:42:58.257460 22136 webserver.cc:460] Webserver started at http://127.8.151.254:32987/ using document root <none> and password file <none>
I20260928 06:42:58.258100 22136 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:58.258179 22136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:42:58.258390 22136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:42:58.261068 22136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "931f79f087b3423f9f1be1a768c2624e"
format_stamp: "Formatted at 2026-09-28 06:42:58 on dist-test-slave-gtr8"
server_key: "5c61f390cb89631124b04e86eda0c31d"
server_key_iv: "a8f492d76d3a720557824be267a1e702"
server_key_version: "encryptionkey@0"
I20260928 06:42:58.261696 22136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "931f79f087b3423f9f1be1a768c2624e"
format_stamp: "Formatted at 2026-09-28 06:42:58 on dist-test-slave-gtr8"
server_key: "5c61f390cb89631124b04e86eda0c31d"
server_key_iv: "a8f492d76d3a720557824be267a1e702"
server_key_version: "encryptionkey@0"
I20260928 06:42:58.266052 22136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:42:58.268910 22151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:58.270335 22136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 06:42:58.270469 22136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "931f79f087b3423f9f1be1a768c2624e"
format_stamp: "Formatted at 2026-09-28 06:42:58 on dist-test-slave-gtr8"
server_key: "5c61f390cb89631124b04e86eda0c31d"
server_key_iv: "a8f492d76d3a720557824be267a1e702"
server_key_version: "encryptionkey@0"
I20260928 06:42:58.270586 22136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:58.293334 22136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:58.296883 22136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:58.297076 22136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:58.305011 22136 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:43263
I20260928 06:42:58.305052 22203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:43263 every 8 connection(s)
I20260928 06:42:58.306116 22136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:42:58.308648 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 22136
I20260928 06:42:58.308807 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:42:58.309162 8799 external_mini_cluster.cc:1428] Setting key 764bd9bae1a3493b0e9a64acc78ae937
I20260928 06:42:58.309162 22204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:42:58.313853 22204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.151.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.253@KRBTEST.COM" created.
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:58.329639 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.316669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53930 (local address 127.8.151.254:43263)
0928 06:42:58.317123 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.317144 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.317341 (+ 197us) server_negotiation.cc:407] Connection header received
0928 06:42:58.318558 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.318565 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.318808 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.319037 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.320070 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.321040 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.322338 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.322862 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.323021 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.325296 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.325301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.325311 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.325330 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.326775 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.327223 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.327229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.327230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.327325 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.327602 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.327605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.327607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.327949 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.328060 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.328414 (+ 354us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.328618 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":129,"threads_started":1}
I20260928 06:42:58.329638 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.316669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:43263 (local address 127.0.0.1:53930)
0928 06:42:58.317174 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.317224 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:58.318200 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:58.319055 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:58.319060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:58.319547 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:58.319884 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.319901 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.321165 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.321168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.322215 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.322226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.323241 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.323244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.323325 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.323820 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:58.323836 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:58.325170 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:58.326870 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.326878 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.326888 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.327100 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.327406 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.327409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.327411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.327515 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.328059 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:58.328066 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:58.328190 (+ 124us) client_negotiation.cc:769] Sending connection context
0928 06:42:58.328324 (+ 134us) client_negotiation.cc:240] Negotiation successful
0928 06:42:58.328707 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":94,"spinlock_wait_cycles":896,"thread_start_us":165,"threads_started":1}
I20260928 06:42:58.332182 22204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:42:58.332888 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.332659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45945 (local address 127.0.0.1:36784)
0928 06:42:58.332766 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.332830 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 06:42:58.333132 22153 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45945: connect: Connection refused (error 111)
I20260928 06:42:58.333946 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.333763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45945 (local address 127.0.0.1:36788)
0928 06:42:58.333854 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.333895 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 06:42:58.334120 22204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.253:45945: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45945: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab.
Entry for principal kudu/127.8.151.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab.
I20260928 06:42:58.337477 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.253:45945
--webserver_interface=127.8.151.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab
--principal=kudu/127.8.151.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:43263,127.8.151.253:45945,127.8.151.252:42111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:42:58.383339 22204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } attempt: 1
I20260928 06:42:58.384136 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.383866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45945 (local address 127.0.0.1:36798)
0928 06:42:58.384001 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.384060 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 06:42:58.384822 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.384624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45945 (local address 127.0.0.1:36810)
0928 06:42:58.384724 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.384767 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 06:42:58.385002 22204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.253:45945: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45945: connect: Connection refused (error 111)
I20260928 06:42:58.453171 22204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } attempt: 2
I20260928 06:42:58.454074 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.453738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45945 (local address 127.0.0.1:36816)
0928 06:42:58.453905 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.453979 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 06:42:58.454931 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.454705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45945 (local address 127.0.0.1:36824)
0928 06:42:58.454810 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.454867 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 06:42:58.455127 22204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.253:45945: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45945: connect: Connection refused (error 111)
W20260928 06:42:58.471818 22211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:58.472090 22211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:58.472133 22211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:58.476402 22211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:42:58.476480 22211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:58.476509 22211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:58.476531 22211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:42:58.476552 22211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:42:58.482060 22211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:43263,127.8.151.253:45945,127.8.151.252:42111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab
--principal=kudu/127.8.151.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.253:45945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.151.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:58.483214 22211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:58.484246 22211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:42:58.492455 22217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:58.492440 22216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:58.492764 22211 server_base.cc:1034] running on GCE node
W20260928 06:42:58.492810 22219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:58.493709 22211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:58.495087 22211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:58.496311 22211 hybrid_clock.cc:648] HybridClock initialized: now 1790577778496280 us; error 38 us; skew 500 ppm
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:58.499538 22211 init.cc:377] Logged in from keytab as kudu/127.8.151.253@KRBTEST.COM (short username kudu)
I20260928 06:42:58.500383 22211 webserver.cc:460] Webserver started at http://127.8.151.253:44911/ using document root <none> and password file <none>
I20260928 06:42:58.500926 22211 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:58.500986 22211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:42:58.501166 22211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:42:58.503206 22211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/data/instance:
uuid: "0cacd2ebf06b42998ad045ce1b62b11b"
format_stamp: "Formatted at 2026-09-28 06:42:58 on dist-test-slave-gtr8"
server_key: "a9beb7b7fc4c0b991ed362a2e6c52135"
server_key_iv: "820de5dd7cc1135455ab8f84257bfec6"
server_key_version: "encryptionkey@0"
I20260928 06:42:58.503801 22211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/wal/instance:
uuid: "0cacd2ebf06b42998ad045ce1b62b11b"
format_stamp: "Formatted at 2026-09-28 06:42:58 on dist-test-slave-gtr8"
server_key: "a9beb7b7fc4c0b991ed362a2e6c52135"
server_key_iv: "820de5dd7cc1135455ab8f84257bfec6"
server_key_version: "encryptionkey@0"
I20260928 06:42:58.507614 22211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260928 06:42:58.510121 22226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:58.511463 22211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:42:58.511605 22211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/wal
uuid: "0cacd2ebf06b42998ad045ce1b62b11b"
format_stamp: "Formatted at 2026-09-28 06:42:58 on dist-test-slave-gtr8"
server_key: "a9beb7b7fc4c0b991ed362a2e6c52135"
server_key_iv: "820de5dd7cc1135455ab8f84257bfec6"
server_key_version: "encryptionkey@0"
I20260928 06:42:58.511710 22211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:58.534134 22211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:58.537287 22211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:58.537470 22211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:58.544858 22211 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.253:45945
I20260928 06:42:58.544903 22278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.253:45945 every 8 connection(s)
I20260928 06:42:58.545828 22211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
I20260928 06:42:58.546303 22204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } attempt: 3
I20260928 06:42:58.549046 22279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:42:58.553469 22279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:42:58.554265 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 22211
I20260928 06:42:58.554409 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-1/wal/instance
I20260928 06:42:58.554718 8799 external_mini_cluster.cc:1428] Setting key 83949d9dd66621b334f94888ccef0b1f
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
I20260928 06:42:58.560300 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.546840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45945 (local address 127.0.0.1:36830)
0928 06:42:58.547033 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.547048 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:58.547178 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:58.549868 (+ 2690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:58.549872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:58.550079 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:58.550234 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.550243 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.552033 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.552036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.552808 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.552822 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.553757 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.553761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.553874 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.554427 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:58.554442 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:58.556003 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:58.558381 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.558384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.558386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.558614 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.558969 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.558972 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.558974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.559068 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.559797 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:58.559801 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:58.559898 (+ 97us) client_negotiation.cc:769] Sending connection context
0928 06:42:58.559998 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 06:42:58.560128 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260928 06:42:58.562259 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.547702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.8.151.253:45945)
0928 06:42:58.548165 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.548183 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.548224 (+ 41us) server_negotiation.cc:407] Connection header received
0928 06:42:58.549049 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.549072 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.549478 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.549893 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.550370 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.551881 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.552946 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.553625 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.553775 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.556254 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.556261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.556278 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.556312 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.558253 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.558736 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.558741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.558747 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.558874 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.559204 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.559207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.559209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.559689 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.559810 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.560708 (+ 898us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.561000 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":134,"threads_started":1}
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:58.564494 22204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:42:58.565237 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.564950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48274)
0928 06:42:58.565124 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.565172 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260928 06:42:58.566016 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.565757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48286)
0928 06:42:58.565907 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.565947 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260928 06:42:58.566206 22204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:42111: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
I20260928 06:42:58.566694 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.556791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53932 (local address 127.8.151.254:43263)
0928 06:42:58.556910 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.556913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.557268 (+ 355us) server_negotiation.cc:407] Connection header received
0928 06:42:58.557454 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.557457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.557498 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.557557 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.558178 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.559356 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.560382 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.560714 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.560786 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.563122 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.563125 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.563128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.563174 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.564764 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.565303 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.565307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.565309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.565392 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.565711 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.565714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.565716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.565925 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.565996 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.566457 (+ 461us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.566570 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 06:42:58.566761 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.555916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:43263 (local address 127.0.0.1:53932)
0928 06:42:58.557146 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.557200 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:58.557358 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:58.557605 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:58.557611 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:58.557941 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:58.558055 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.558061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.559459 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.559462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.560254 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.560264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.560922 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.560925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.561035 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.561508 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:58.561522 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:58.562990 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:58.564888 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.564898 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.564904 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.565185 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.565504 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.565507 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.565509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.565605 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.566085 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:58.566091 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:58.566223 (+ 132us) client_negotiation.cc:769] Sending connection context
0928 06:42:58.566377 (+ 154us) client_negotiation.cc:240] Negotiation successful
0928 06:42:58.566581 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1076,"thread_start_us":159,"threads_started":1}
WARNING: no policy specified for kudu/127.8.151.252@KRBTEST.COM; defaulting to no policy
I20260928 06:42:58.567873 22279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.252@KRBTEST.COM" created.
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
I20260928 06:42:58.575655 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.568307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45945 (local address 127.0.0.1:36840)
0928 06:42:58.568398 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.568410 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:58.568479 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:58.568708 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:58.568710 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:58.568861 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:58.568971 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.568977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.569799 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.569802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.570305 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.570312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.570760 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.570763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.570847 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.571266 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:58.571281 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:58.572606 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:58.574065 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.574068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.574070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.574257 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.574513 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.574515 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.574517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.574605 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.575090 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:58.575093 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:58.575207 (+ 114us) client_negotiation.cc:769] Sending connection context
0928 06:42:58.575327 (+ 120us) client_negotiation.cc:240] Negotiation successful
0928 06:42:58.575456 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260928 06:42:58.575654 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.568362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36840 (local address 127.8.151.253:45945)
0928 06:42:58.568457 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.568461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.568472 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:42:58.568548 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.568550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.568624 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.568683 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.569069 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.569714 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.570402 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.570665 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.570748 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.572720 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.572723 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.572725 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.572744 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.573969 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.574346 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.574349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.574351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.574423 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.574693 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.574696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.574697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.574900 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.574971 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.575411 (+ 440us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.575506 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":35}
I20260928 06:42:58.577080 22279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:42:58.577759 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.577497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48296)
0928 06:42:58.577608 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.577700 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260928 06:42:58.577955 22229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
I20260928 06:42:58.578755 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.578552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48300)
0928 06:42:58.578649 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.578696 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260928 06:42:58.578890 22279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:42111: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab.
Entry for principal kudu/127.8.151.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab.
I20260928 06:42:58.586375 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.252:42111
--webserver_interface=127.8.151.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab
--principal=kudu/127.8.151.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:43263,127.8.151.253:45945,127.8.151.252:42111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:42:58.597373 22204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } attempt: 1
I20260928 06:42:58.598155 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.597870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48312)
0928 06:42:58.598011 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.598079 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 06:42:58.598816 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.598620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48326)
0928 06:42:58.598717 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.598762 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 06:42:58.599076 22204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:42111: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
I20260928 06:42:58.628160 22279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } attempt: 1
I20260928 06:42:58.629004 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.628720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48328)
0928 06:42:58.628864 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.628925 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 06:42:58.629690 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.629490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48330)
0928 06:42:58.629588 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.629633 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 06:42:58.629856 22279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:42111: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
I20260928 06:42:58.674294 22204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } attempt: 2
I20260928 06:42:58.675239 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.674842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48344)
0928 06:42:58.675023 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.675105 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260928 06:42:58.676111 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.675877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48360)
0928 06:42:58.675970 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.676036 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260928 06:42:58.676326 22204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:42111: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
I20260928 06:42:58.698010 22279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } attempt: 2
I20260928 06:42:58.698884 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.698567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48374)
0928 06:42:58.698727 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.698796 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 06:42:58.699678 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.699441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48390)
0928 06:42:58.699554 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.699607 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260928 06:42:58.699903 22279 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:42111: Network error: Client connection negotiation failed: client connection to 127.8.151.252:42111: connect: Connection refused (error 111)
W20260928 06:42:58.708395 22286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:42:58.708642 22286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:42:58.708688 22286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:42:58.712646 22286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:42:58.712695 22286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:42:58.712714 22286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:42:58.712729 22286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:42:58.712761 22286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:42:58.718052 22286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:43263,127.8.151.253:45945,127.8.151.252:42111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab
--principal=kudu/127.8.151.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.252:42111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.151.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:42:58.719178 22286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:42:58.720248 22286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:42:58.728847 22286 server_base.cc:1034] running on GCE node
W20260928 06:42:58.728726 22291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:58.728726 22292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:42:58.729032 22294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:42:58.729573 22286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:42:58.730631 22286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:42:58.731789 22286 hybrid_clock.cc:648] HybridClock initialized: now 1790577778731769 us; error 36 us; skew 500 ppm
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:42:58.734567 22286 init.cc:377] Logged in from keytab as kudu/127.8.151.252@KRBTEST.COM (short username kudu)
I20260928 06:42:58.735472 22286 webserver.cc:460] Webserver started at http://127.8.151.252:43041/ using document root <none> and password file <none>
I20260928 06:42:58.735980 22286 fs_manager.cc:359] Metadata directory not provided
I20260928 06:42:58.736059 22286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:42:58.736258 22286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:42:58.738224 22286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/data/instance:
uuid: "e4d0fac2906146e0a1ef3591040e48c4"
format_stamp: "Formatted at 2026-09-28 06:42:58 on dist-test-slave-gtr8"
server_key: "ad54a9ef116240f02a3841891244942a"
server_key_iv: "b172c05291adb7d24e9319ffbf1e2825"
server_key_version: "encryptionkey@0"
I20260928 06:42:58.738740 22286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/wal/instance:
uuid: "e4d0fac2906146e0a1ef3591040e48c4"
format_stamp: "Formatted at 2026-09-28 06:42:58 on dist-test-slave-gtr8"
server_key: "ad54a9ef116240f02a3841891244942a"
server_key_iv: "b172c05291adb7d24e9319ffbf1e2825"
server_key_version: "encryptionkey@0"
I20260928 06:42:58.742405 22286 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260928 06:42:58.744808 22301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:42:58.746053 22286 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:42:58.746163 22286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/wal
uuid: "e4d0fac2906146e0a1ef3591040e48c4"
format_stamp: "Formatted at 2026-09-28 06:42:58 on dist-test-slave-gtr8"
server_key: "ad54a9ef116240f02a3841891244942a"
server_key_iv: "b172c05291adb7d24e9319ffbf1e2825"
server_key_version: "encryptionkey@0"
I20260928 06:42:58.746255 22286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:42:58.760759 22286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:42:58.763701 22286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:42:58.763867 22286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:42:58.770965 22286 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.252:42111
I20260928 06:42:58.770994 22353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.252:42111 every 8 connection(s)
I20260928 06:42:58.771926 22286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
I20260928 06:42:58.772727 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 22286
I20260928 06:42:58.772859 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790577605970741-8799-0/minicluster-data/master-2/wal/instance
I20260928 06:42:58.773142 8799 external_mini_cluster.cc:1428] Setting key 877e83c53b486ada00126ba3386ebe00
I20260928 06:42:58.775467 22204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } attempt: 3
I20260928 06:42:58.776041 22354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:42:58.780843 22354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
I20260928 06:42:58.788997 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.775913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48414)
0928 06:42:58.776174 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.776187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:58.776262 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:58.778277 (+ 2015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:58.778280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:58.778460 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:58.778568 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.778577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.779753 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.779757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.780497 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.780509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.781508 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.781512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.781593 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.782126 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:58.782141 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:58.783708 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:58.786630 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.786634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.786636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.786866 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.787282 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.787285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.787287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.787378 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.788517 (+ 1139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:58.788521 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:58.788622 (+ 101us) client_negotiation.cc:769] Sending connection context
0928 06:42:58.788705 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 06:42:58.788829 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:58.791049 22279 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } attempt: 3
I20260928 06:42:58.792676 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.776412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.8.151.252:42111)
0928 06:42:58.778032 (+ 1620us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.778053 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.778066 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:42:58.778097 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.778099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.778135 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.778679 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.778804 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.779618 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.780640 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.781380 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.781857 (+ 477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.783896 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.783902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.783916 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.783943 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.786510 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.786986 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.786991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.786997 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.787180 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.787475 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.787478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.787480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.788131 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.789316 (+ 1185us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.789400 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.792312 (+ 2912us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2875,"server-negotiator.queue_time_us":1556,"spinlock_wait_cycles":5504,"thread_start_us":76,"threads_started":1}
I20260928 06:42:58.792682 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.774384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48404 (local address 127.8.151.252:42111)
0928 06:42:58.774774 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.774793 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.774836 (+ 43us) server_negotiation.cc:407] Connection header received
0928 06:42:58.775771 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.775794 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.776286 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.776633 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.777940 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.779441 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.780550 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.781334 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.783352 (+ 2018us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.784395 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.784399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.784401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.784431 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.786534 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.787001 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.787004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.787006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.787192 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.787487 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.787490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.787491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.787988 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.788883 (+ 895us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.789053 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.790343 (+ 1290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":257,"spinlock_wait_cycles":896,"thread_start_us":116,"threads_started":1}
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
I20260928 06:42:58.798034 22204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e: Bootstrap starting.
I20260928 06:42:58.798401 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.783257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53946 (local address 127.8.151.254:43263)
0928 06:42:58.783887 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.783891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.784283 (+ 392us) server_negotiation.cc:407] Connection header received
0928 06:42:58.784470 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.784473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.784516 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.784604 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.785525 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.786255 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.787441 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.787759 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.787831 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.791351 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.791355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.791358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.791383 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.795412 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.795967 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.795970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.795972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.796062 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.796625 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.796629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.796630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.796888 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.796969 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.798137 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.798255 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555}
I20260928 06:42:58.799171 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.782740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:43263 (local address 127.0.0.1:53946)
0928 06:42:58.784171 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.784224 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:58.784364 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:58.784757 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:58.784761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:58.785250 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:58.785391 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.785400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.786391 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.786395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.787322 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.787330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.787963 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.787966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.788057 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.788598 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:58.788611 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:58.791175 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:58.795531 (+ 4356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.795553 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.795559 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.795855 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.796158 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.796161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.796163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.796247 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.797505 (+ 1258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:58.797511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:58.797902 (+ 391us) client_negotiation.cc:769] Sending connection context
0928 06:42:58.798771 (+ 869us) client_negotiation.cc:240] Negotiation successful
0928 06:42:58.798970 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1252,"thread_start_us":80,"threads_started":1}
I20260928 06:42:58.800488 22354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:42:58.800889 22204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e: Neither blocks nor log segments found. Creating new log.
I20260928 06:42:58.802026 22204 log.cc:824] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:58.804564 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.791527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48426)
0928 06:42:58.791798 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.791811 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:58.791889 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:58.793331 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:58.793334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:58.793509 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:58.793697 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.793705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.794653 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.794657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.795502 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.795515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.796090 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.796095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.796182 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.796581 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:58.796595 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:58.800292 (+ 3697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:58.803072 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.803076 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.803078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.803358 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.803646 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.803649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.803650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.803731 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.804126 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:58.804129 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:58.804217 (+ 88us) client_negotiation.cc:769] Sending connection context
0928 06:42:58.804285 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 06:42:58.804411 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:42:58.805160 22360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.792020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48426 (local address 127.8.151.252:42111)
0928 06:42:58.793084 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.793092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.793106 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:42:58.793138 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.793141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.793183 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.793807 (+ 624us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.793920 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.794532 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.795621 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.795957 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.796379 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.800801 (+ 4422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.800804 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.800807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.800829 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.802943 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.803465 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.803468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.803470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.803544 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.803827 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.803830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.803831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.804015 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.804821 (+ 806us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.804891 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.804998 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":139,"threads_started":1}
I20260928 06:42:58.805238 22204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e: No bootstrap required, opened a new log
I20260928 06:42:58.808064 22279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b: Bootstrap starting.
I20260928 06:42:58.809218 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.798629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53950 (local address 127.8.151.254:43263)
0928 06:42:58.798799 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.798803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.798814 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:42:58.798844 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.798846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.798884 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.798951 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.799948 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.801040 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.802486 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.802830 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.802937 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.805521 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.805525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.805528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.805553 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.807045 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.807563 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.807566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.807568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.807651 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.807929 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.807933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.807934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.808145 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.808298 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.808963 (+ 665us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.809084 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260928 06:42:58.809391 22204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } }
I20260928 06:42:58.809610 22204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:42:58.809659 22204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f79f087b3423f9f1be1a768c2624e, State: Initialized, Role: FOLLOWER
I20260928 06:42:58.810109 22204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } }
I20260928 06:42:58.810657 22279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b: Neither blocks nor log segments found. Creating new log.
I20260928 06:42:58.811488 22279 log.cc:824] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:58.811535 22204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:42:58.813791 22362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } } }
I20260928 06:42:58.813928 22362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:42:58.815891 22279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b: No bootstrap required, opened a new log
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
I20260928 06:42:58.818046 22362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:42:58.818181 22362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } }
I20260928 06:42:58.819365 22362 leader_election.cc:288] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cacd2ebf06b42998ad045ce1b62b11b (127.8.151.253:45945), e4d0fac2906146e0a1ef3591040e48c4 (127.8.151.252:42111)
I20260928 06:42:58.820827 22279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } }
I20260928 06:42:58.821045 22279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:42:58.821094 22279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cacd2ebf06b42998ad045ce1b62b11b, State: Initialized, Role: FOLLOWER
I20260928 06:42:58.816305 22136 master_runner.cc:186] Error getting master registration for 127.8.151.253:45945: OK, instance_id { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" instance_seqno: 1790577778542737 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 06:42:58.821563 22136 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 06:42:58.821615 22279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } }
I20260928 06:42:58.823010 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.810485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.8.151.253:45945)
0928 06:42:58.811114 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.811123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.811155 (+ 32us) server_negotiation.cc:407] Connection header received
0928 06:42:58.811192 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.811194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.811239 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.812059 (+ 820us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.812165 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.812933 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.813948 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.814295 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.817248 (+ 2953us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.817382 (+ 134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.817386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.817389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.817409 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.820672 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.821184 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.821187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.821189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.821270 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.821617 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.821620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.821622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.821867 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.822657 (+ 790us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.822731 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.822844 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":98,"threads_started":1}
I20260928 06:42:58.823086 22253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "931f79f087b3423f9f1be1a768c2624e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" is_pre_election: true
I20260928 06:42:58.823392 22253 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f79f087b3423f9f1be1a768c2624e in term 0.
I20260928 06:42:58.823508 22376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } } }
I20260928 06:42:58.823642 22376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:42:58.823014 22279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:42:58.824517 22153 leader_election.cc:302] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cacd2ebf06b42998ad045ce1b62b11b, 931f79f087b3423f9f1be1a768c2624e; no voters:
I20260928 06:42:58.824805 22362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:42:58.824883 22362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:42:58.824925 22362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:42:58.826177 22362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } }
I20260928 06:42:58.826539 22362 leader_election.cc:288] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [CANDIDATE]: Term 1 election: Requested vote from peers 0cacd2ebf06b42998ad045ce1b62b11b (127.8.151.253:45945), e4d0fac2906146e0a1ef3591040e48c4 (127.8.151.252:42111)
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
I20260928 06:42:58.829351 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260928 06:42:58.829480 22387 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:42:58.829605 22387 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:42:58.832906 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.801710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36846 (local address 127.8.151.253:45945)
0928 06:42:58.801836 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.801841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.803172 (+ 1331us) server_negotiation.cc:407] Connection header received
0928 06:42:58.803296 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.803299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.803351 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.803428 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.816626 (+ 13198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.817300 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.824098 (+ 6798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.824411 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.824488 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.827195 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.827199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.827201 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.827225 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.828744 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.831593 (+ 2849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.831597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.831599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.831684 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.831985 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.831989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.831990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.832211 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.832292 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.832645 (+ 353us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.832762 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 06:42:58.833369 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.801305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45945 (local address 127.0.0.1:36846)
0928 06:42:58.803084 (+ 1779us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.803097 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:58.803197 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:58.816190 (+ 12993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:58.816194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:58.816377 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:58.816490 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.816499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.823077 (+ 6578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.823081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.823948 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.823961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.824665 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.824669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.824768 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.825552 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:58.825568 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:58.827024 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:58.831211 (+ 4187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.831215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.831217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.831470 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.831792 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.831795 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.831796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.831883 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.832349 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:58.832353 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:58.832454 (+ 101us) client_negotiation.cc:769] Sending connection context
0928 06:42:58.833065 (+ 611us) client_negotiation.cc:240] Negotiation successful
0928 06:42:58.833223 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1710}
W20260928 06:42:58.834168 22375 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:42:58.834241 22375 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:42:58.831759 22211 master_runner.cc:186] Error getting master registration for 127.8.151.252:42111: OK, instance_id { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" instance_seqno: 1790577778768916 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 06:42:58.834908 22211 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 06:42:58.835410 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "931f79f087b3423f9f1be1a768c2624e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cacd2ebf06b42998ad045ce1b62b11b"
I20260928 06:42:58.835544 22252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:42:58.836794 22252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f79f087b3423f9f1be1a768c2624e in term 1.
I20260928 06:42:58.837335 22354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:42:58.838248 22153 leader_election.cc:302] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cacd2ebf06b42998ad045ce1b62b11b, 931f79f087b3423f9f1be1a768c2624e; no voters:
I20260928 06:42:58.838464 22362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:42:58.838699 22362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [term 1 LEADER]: Becoming Leader. State: Replica: 931f79f087b3423f9f1be1a768c2624e, State: Running, Role: LEADER
I20260928 06:42:58.839053 22362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } }
I20260928 06:42:58.840890 22362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 931f79f087b3423f9f1be1a768c2624e. Latest consensus state: current_term: 1 leader_uuid: "931f79f087b3423f9f1be1a768c2624e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } } }
I20260928 06:42:58.841010 22362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [sys.catalog]: This master's current role is: LEADER
I20260928 06:42:58.845824 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.831712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53960 (local address 127.8.151.254:43263)
0928 06:42:58.831822 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.831826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.834267 (+ 2441us) server_negotiation.cc:407] Connection header received
0928 06:42:58.834550 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.834553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.834601 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.834650 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.836070 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.836803 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.838318 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.838619 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.838688 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.840963 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.840967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.840969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.840996 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.842837 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.843784 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.843788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.843803 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.844118 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.844522 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.844525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.844527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.844728 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.844842 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.845564 (+ 722us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.845680 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Sep 28 06:42:58 dist-test-slave-gtr8 krb5kdc[22123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577778, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
I20260928 06:42:58.846681 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.835035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48430 (local address 127.8.151.252:42111)
0928 06:42:58.838016 (+ 2981us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.838021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.838033 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:42:58.838066 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.838069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.838106 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.838169 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.839313 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.840017 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.841130 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.841464 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.841535 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.843108 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.843111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.843114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.843156 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.844664 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.845098 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.845102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.845103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.845187 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.845477 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.845480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.845482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.845676 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.845802 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.846418 (+ 616us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.846527 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2904}
I20260928 06:42:58.850189 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.837761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:42111 (local address 127.0.0.1:48436)
0928 06:42:58.838013 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0928 06:42:58.838025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:42:58.838104 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:42:58.841599 (+ 3495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:42:58.841602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:42:58.841765 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:42:58.841853 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.841859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.843278 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.843281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.844023 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:42:58.844035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.844796 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.844800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:42:58.844890 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.845272 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:42:58.845285 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:42:58.846655 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:42:58.848634 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.848637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.848639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.848853 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.849212 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:42:58.849215 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:42:58.849217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:42:58.849285 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:42:58.849776 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:42:58.849778 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:42:58.849860 (+ 82us) client_negotiation.cc:769] Sending connection context
0928 06:42:58.849934 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 06:42:58.850043 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260928 06:42:58.850813 22360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.837923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.8.151.252:42111)
0928 06:42:58.841347 (+ 3424us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.841351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.841363 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:42:58.841412 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.841415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.841476 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.841960 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.842092 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.843169 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.844201 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.844662 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.845090 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.846796 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.846820 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.846823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.846843 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.848531 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.848965 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.848968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.848969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.849099 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.849373 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.849376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.849377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.849648 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.850336 (+ 688us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.850478 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.850623 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3314}
I20260928 06:42:58.853863 22354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4: Bootstrap starting.
I20260928 06:42:58.855686 22399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:42:58.856608 22354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4: Neither blocks nor log segments found. Creating new log.
I20260928 06:42:58.856843 22399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:42:58.857497 22354 log.cc:824] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4: Log is configured to *not* fsync() on all Append() calls
I20260928 06:42:58.860611 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:42:58.833368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.8.151.253:45945)
0928 06:42:58.833498 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 06:42:58.833502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:42:58.834991 (+ 1489us) server_negotiation.cc:407] Connection header received
0928 06:42:58.835171 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:42:58.835174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:42:58.835217 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:42:58.835266 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:42:58.836028 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.836802 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.855356 (+ 18554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:42:58.855682 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:42:58.855787 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:42:58.857268 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:42:58.857272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:42:58.857275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:42:58.857304 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:42:58.858950 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.859417 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.859421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.859438 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.859521 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:42:58.859782 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:42:58.859785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:42:58.859787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:42:58.860000 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:42:58.860145 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 06:42:58.860360 (+ 215us) server_negotiation.cc:299] Negotiation successful
0928 06:42:58.860466 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:42:58.871181 22354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4: No bootstrap required, opened a new log
I20260928 06:42:58.871377 22253 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [term 1 FOLLOWER]: Refusing update from remote peer 931f79f087b3423f9f1be1a768c2624e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:42:58.872041 22397 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:42:58.877652 22354 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } }
I20260928 06:42:58.878083 22376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 931f79f087b3423f9f1be1a768c2624e. Latest consensus state: current_term: 1 leader_uuid: "931f79f087b3423f9f1be1a768c2624e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } } }
I20260928 06:42:58.878194 22376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:42:58.878170 22354 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:42:58.878371 22354 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4d0fac2906146e0a1ef3591040e48c4, State: Initialized, Role: FOLLOWER
I20260928 06:42:58.879204 22354 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } }
I20260928 06:42:58.881501 22327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "931f79f087b3423f9f1be1a768c2624e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4d0fac2906146e0a1ef3591040e48c4"
I20260928 06:42:58.881723 22326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:42:58.881521 22328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "931f79f087b3423f9f1be1a768c2624e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4d0fac2906146e0a1ef3591040e48c4" is_pre_election: true
I20260928 06:42:58.881443 22354 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:42:58.883062 22326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [term 1 FOLLOWER]: Refusing update from remote peer 931f79f087b3423f9f1be1a768c2624e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:42:58.884027 22397 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:42:58.891911 22416 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "931f79f087b3423f9f1be1a768c2624e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } } }
I20260928 06:42:58.892041 22416 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:42:58.892122 22416 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 931f79f087b3423f9f1be1a768c2624e. Latest consensus state: current_term: 1 leader_uuid: "931f79f087b3423f9f1be1a768c2624e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } } }
I20260928 06:42:58.892191 22416 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:42:58.894722 22362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "931f79f087b3423f9f1be1a768c2624e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } } }
I20260928 06:42:58.894852 22362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [sys.catalog]: This master's current role is: LEADER
I20260928 06:42:58.895495 22376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "931f79f087b3423f9f1be1a768c2624e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } } }
I20260928 06:42:58.895617 22376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:42:58.895973 22401 mvcc.cc:205] Tried to move back new op lower bound from 7334206582246117376 to 7334206582128295936. Current Snapshot: MvccSnapshot[applied={T|T < 7334206582246117376}]
I20260928 06:42:58.896801 22402 mvcc.cc:205] Tried to move back new op lower bound from 7334206582246117376 to 7334206582128295936. Current Snapshot: MvccSnapshot[applied={T|T < 7334206582246117376}]
I20260928 06:42:58.898581 22397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "931f79f087b3423f9f1be1a768c2624e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } } }
I20260928 06:42:58.898700 22397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e [sys.catalog]: This master's current role is: LEADER
I20260928 06:42:58.899755 22416 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "931f79f087b3423f9f1be1a768c2624e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f79f087b3423f9f1be1a768c2624e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43263 } } peers { permanent_uuid: "0cacd2ebf06b42998ad045ce1b62b11b" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45945 } } peers { permanent_uuid: "e4d0fac2906146e0a1ef3591040e48c4" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 42111 } } }
I20260928 06:42:58.899866 22416 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:42:58.900147 22417 mvcc.cc:205] Tried to move back new op lower bound from 7334206582246117376 to 7334206582128295936. Current Snapshot: MvccSnapshot[applied={T|T < 7334206582246117376}]
W20260928 06:42:58.907907 22415 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:42:58.908008 22415 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:42:58.912983 22399 catalog_manager.cc:1347] Generated new cluster ID: a78a4535c1794d66bf9fee21615c7ab0
I20260928 06:42:58.913071 22399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:42:58.964938 22399 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:42:58.966272 22399 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:42:58.987218 22399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 931f79f087b3423f9f1be1a768c2624e: Generated new TSK 0
I20260928 06:42:58.988143 22399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:42:59.830904 22387 catalog_manager.cc:1259] Loaded cluster ID: a78a4535c1794d66bf9fee21615c7ab0
I20260928 06:42:59.830994 22387 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b: loading cluster ID for follower catalog manager: success
I20260928 06:42:59.832769 22387 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b: acquiring CA information for follower catalog manager: success
I20260928 06:42:59.833952 22387 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0cacd2ebf06b42998ad045ce1b62b11b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 06:42:59.909118 22415 catalog_manager.cc:1259] Loaded cluster ID: a78a4535c1794d66bf9fee21615c7ab0
I20260928 06:42:59.909204 22415 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4: loading cluster ID for follower catalog manager: success
I20260928 06:42:59.910845 22415 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4: acquiring CA information for follower catalog manager: success
I20260928 06:42:59.912118 22415 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e4d0fac2906146e0a1ef3591040e48c4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 06:43:00.073200 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 22136
I20260928 06:43:00.080750 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 22211
I20260928 06:43:00.086160 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 22286
2026-09-28T06:43:00Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3056 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:43:00 dist-test-slave-gtr8 krb5kdc[22428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:43:00 dist-test-slave-gtr8 krb5kdc[22428](info): set up 2 sockets
Sep 28 06:43:00 dist-test-slave-gtr8 krb5kdc[22428](info): commencing operation
krb5kdc: starting...
W20260928 06:43:01.043530 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:43:01 dist-test-slave-gtr8 krb5kdc[22428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:43:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:43:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:43:01.162683 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39765
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:39765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:01.281163 22441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:01.281427 22441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:01.281467 22441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:01.285459 22441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:43:01.285516 22441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:01.285534 22441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:01.285548 22441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:43:01.285562 22441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:43:01.290738 22441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:39765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:01.291800 22441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:01.292797 22441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:01.300992 22449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:01.301010 22447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:01.300993 22446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:01.301241 22441 server_base.cc:1034] running on GCE node
I20260928 06:43:01.301887 22441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:01.303046 22441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:01.304226 22441 hybrid_clock.cc:648] HybridClock initialized: now 1790577781304203 us; error 40 us; skew 500 ppm
Sep 28 06:43:01 dist-test-slave-gtr8 krb5kdc[22428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577781, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:01.306952 22441 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:43:01.307920 22441 webserver.cc:460] Webserver started at http://127.8.151.254:43459/ using document root <none> and password file <none>
I20260928 06:43:01.308476 22441 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:01.308555 22441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:01.308760 22441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:43:01.310765 22441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "aba3ad76242b4ae79f6eeea5ba3febb6"
format_stamp: "Formatted at 2026-09-28 06:43:01 on dist-test-slave-gtr8"
server_key: "2a75988146e629dbb9b0ca6e1bbe36fd"
server_key_iv: "2f3aa43ddf213eb16a7d454a8c186ace"
server_key_version: "encryptionkey@0"
I20260928 06:43:01.311340 22441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "aba3ad76242b4ae79f6eeea5ba3febb6"
format_stamp: "Formatted at 2026-09-28 06:43:01 on dist-test-slave-gtr8"
server_key: "2a75988146e629dbb9b0ca6e1bbe36fd"
server_key_iv: "2f3aa43ddf213eb16a7d454a8c186ace"
server_key_version: "encryptionkey@0"
I20260928 06:43:01.314961 22441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260928 06:43:01.317354 22456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:01.318591 22441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:43:01.318701 22441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "aba3ad76242b4ae79f6eeea5ba3febb6"
format_stamp: "Formatted at 2026-09-28 06:43:01 on dist-test-slave-gtr8"
server_key: "2a75988146e629dbb9b0ca6e1bbe36fd"
server_key_iv: "2f3aa43ddf213eb16a7d454a8c186ace"
server_key_version: "encryptionkey@0"
I20260928 06:43:01.318813 22441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:01.331441 22441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:01.334640 22441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:01.334805 22441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:01.342005 22441 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:39765
I20260928 06:43:01.342036 22508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:39765 every 8 connection(s)
I20260928 06:43:01.342993 22441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:43:01.346601 22509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:01.349190 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 22441
I20260928 06:43:01.349305 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:43:01.349560 8799 external_mini_cluster.cc:1428] Setting key 005fb2ab6ccc03f1939ae04431941cd7
I20260928 06:43:01.354230 22509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6: Bootstrap starting.
I20260928 06:43:01.357149 22509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6: Neither blocks nor log segments found. Creating new log.
Sep 28 06:43:01 dist-test-slave-gtr8 krb5kdc[22428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:01.358036 22509 log.cc:824] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:01.360438 22509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6: No bootstrap required, opened a new log
I20260928 06:43:01.363867 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:01.350932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.8.151.254:39765)
0928 06:43:01.351343 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:01.351359 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:01.351395 (+ 36us) server_negotiation.cc:407] Connection header received
0928 06:43:01.352021 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:01.352037 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:01.352384 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:01.352697 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:01.353501 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:01.354535 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:01.355408 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:01.355882 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:01.356028 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:01.358378 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:01.358383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:01.358397 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:01.358427 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:01.360421 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:01.360921 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:01.360927 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:01.360933 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:01.361074 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:01.361376 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:01.361379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:01.361380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:01.361872 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:01.362054 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:01.362348 (+ 294us) server_negotiation.cc:299] Negotiation successful
0928 06:43:01.362862 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":133,"threads_started":1}
I20260928 06:43:01.364936 22509 raft_consensus.cc:348] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba3ad76242b4ae79f6eeea5ba3febb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39765 } }
I20260928 06:43:01.365203 22509 raft_consensus.cc:374] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:01.365276 22509 raft_consensus.cc:729] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba3ad76242b4ae79f6eeea5ba3febb6, State: Initialized, Role: FOLLOWER
I20260928 06:43:01.365731 22509 consensus_queue.cc:260] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba3ad76242b4ae79f6eeea5ba3febb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39765 } }
I20260928 06:43:01.365908 22509 raft_consensus.cc:388] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:43:01.366264 22509 raft_consensus.cc:482] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:43:01.366380 22509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:01.367000 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:43:01.367499 22509 raft_consensus.cc:504] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba3ad76242b4ae79f6eeea5ba3febb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39765 } }
I20260928 06:43:01.367833 22509 leader_election.cc:302] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aba3ad76242b4ae79f6eeea5ba3febb6; no voters:
I20260928 06:43:01.368157 22509 leader_election.cc:288] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:43:01.368558 22515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:43:01.368810 22515 raft_consensus.cc:686] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [term 1 LEADER]: Becoming Leader. State: Replica: aba3ad76242b4ae79f6eeea5ba3febb6, State: Running, Role: LEADER
I20260928 06:43:01.369131 22515 consensus_queue.cc:237] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba3ad76242b4ae79f6eeea5ba3febb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39765 } }
I20260928 06:43:01.369470 22509 sys_catalog.cc:564] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:43:01.370920 22514 sys_catalog.cc:455] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aba3ad76242b4ae79f6eeea5ba3febb6. Latest consensus state: current_term: 1 leader_uuid: "aba3ad76242b4ae79f6eeea5ba3febb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba3ad76242b4ae79f6eeea5ba3febb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39765 } } }
I20260928 06:43:01.370920 22516 sys_catalog.cc:455] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aba3ad76242b4ae79f6eeea5ba3febb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba3ad76242b4ae79f6eeea5ba3febb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39765 } } }
I20260928 06:43:01.371084 22514 sys_catalog.cc:458] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:01.371084 22516 sys_catalog.cc:458] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6 [sys.catalog]: This master's current role is: LEADER
W20260928 06:43:01.374480 22529 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:43:01.374558 22529 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:43:01.374694 22528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:43:01.375633 22528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:43:01.381705 22528 catalog_manager.cc:1347] Generated new cluster ID: 78fca5a1ab914c5aa4365b0ff599d159
I20260928 06:43:01.381773 22528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:43:01.418007 22528 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:43:01.419240 22528 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:43:01.426564 22528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aba3ad76242b4ae79f6eeea5ba3febb6: Generated new TSK 0
I20260928 06:43:01.427358 22528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:43:01.474472 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 22441
2026-09-28T06:43:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1387 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:43:01 dist-test-slave-gtr8 krb5kdc[22536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:43:01 dist-test-slave-gtr8 krb5kdc[22536](info): set up 2 sockets
Sep 28 06:43:01 dist-test-slave-gtr8 krb5kdc[22536](info): commencing operation
krb5kdc: starting...
W20260928 06:43:02.435483 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:43:02 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:43:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:43:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:43:02.554194 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:45013
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:45013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:02.679549 22549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:02.679847 22549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:02.679898 22549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:02.683981 22549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:43:02.684042 22549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:02.684060 22549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:02.684077 22549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:43:02.684092 22549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:43:02.689393 22549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:45013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:45013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:02.690497 22549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:02.691565 22549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:02.700714 22554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:02.700796 22549 server_base.cc:1034] running on GCE node
W20260928 06:43:02.700714 22555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:02.701006 22557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:02.701622 22549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:02.702890 22549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:02.704067 22549 hybrid_clock.cc:648] HybridClock initialized: now 1790577782704040 us; error 36 us; skew 500 ppm
Sep 28 06:43:02 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577782, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:02.711560 22549 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:43:02.712615 22549 webserver.cc:460] Webserver started at http://127.8.151.254:42739/ using document root <none> and password file <none>
I20260928 06:43:02.713244 22549 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:02.713323 22549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:02.713542 22549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:43:02.716182 22549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "90d8dfb9ad58457c94e9c1fbd0132052"
format_stamp: "Formatted at 2026-09-28 06:43:02 on dist-test-slave-gtr8"
server_key: "797658db8a037efbc34b64bd3a09d745"
server_key_iv: "b1ac740d255853e95edd9b8aaa913cfb"
server_key_version: "encryptionkey@0"
I20260928 06:43:02.716815 22549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "90d8dfb9ad58457c94e9c1fbd0132052"
format_stamp: "Formatted at 2026-09-28 06:43:02 on dist-test-slave-gtr8"
server_key: "797658db8a037efbc34b64bd3a09d745"
server_key_iv: "b1ac740d255853e95edd9b8aaa913cfb"
server_key_version: "encryptionkey@0"
I20260928 06:43:02.721277 22549 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:43:02.724093 22564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:02.725534 22549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260928 06:43:02.725649 22549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "90d8dfb9ad58457c94e9c1fbd0132052"
format_stamp: "Formatted at 2026-09-28 06:43:02 on dist-test-slave-gtr8"
server_key: "797658db8a037efbc34b64bd3a09d745"
server_key_iv: "b1ac740d255853e95edd9b8aaa913cfb"
server_key_version: "encryptionkey@0"
I20260928 06:43:02.725760 22549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:02.761643 22549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:02.764957 22549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:02.765126 22549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:02.771804 22549 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:45013
I20260928 06:43:02.772091 22616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:45013 every 8 connection(s)
I20260928 06:43:02.772806 22549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:43:02.775445 22617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:02.780753 22617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Bootstrap starting.
I20260928 06:43:02.781646 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 22549
I20260928 06:43:02.781778 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:43:02.782063 8799 external_mini_cluster.cc:1428] Setting key 535c72f1a02954d1e9614e971023fd6f
I20260928 06:43:02.783561 22617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:02.784430 22617 log.cc:824] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:02.786583 22617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: No bootstrap required, opened a new log
I20260928 06:43:02.790531 22617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:02.790756 22617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:02.790813 22617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d8dfb9ad58457c94e9c1fbd0132052, State: Initialized, Role: FOLLOWER
I20260928 06:43:02.791271 22617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:02.791419 22617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:43:02.791487 22617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:43:02.791584 22617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 0 FOLLOWER]: Advancing to term 1
Sep 28 06:43:02 dist-test-slave-gtr8 krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:02.792680 22617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:02.793044 22617 leader_election.cc:302] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90d8dfb9ad58457c94e9c1fbd0132052; no voters:
I20260928 06:43:02.793378 22617 leader_election.cc:288] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:43:02.793689 22622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:43:02.793867 22622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 1 LEADER]: Becoming Leader. State: Replica: 90d8dfb9ad58457c94e9c1fbd0132052, State: Running, Role: LEADER
I20260928 06:43:02.794149 22622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:02.794667 22617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:43:02.795361 22622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } } }
I20260928 06:43:02.795504 22622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:02.795686 22622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90d8dfb9ad58457c94e9c1fbd0132052. Latest consensus state: current_term: 1 leader_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } } }
I20260928 06:43:02.795778 22622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:02.796033 22630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:43:02.798502 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:02.783235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.8.151.254:45013)
0928 06:43:02.783733 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:02.783762 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:02.783809 (+ 47us) server_negotiation.cc:407] Connection header received
0928 06:43:02.784692 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:02.784715 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:02.785076 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:02.785384 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:02.786197 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:02.787656 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:02.788801 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:02.789546 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:02.789700 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:02.792556 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:02.792563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:02.792582 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:02.792616 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:02.794679 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:02.795252 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:02.795257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:02.795263 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:02.795393 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:02.795839 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:02.795843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:02.795845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:02.796387 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:02.797123 (+ 736us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:02.797280 (+ 157us) server_negotiation.cc:299] Negotiation successful
0928 06:43:02.797565 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":130,"threads_started":1}
I20260928 06:43:02.799434 22630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:43:02.805864 22630 catalog_manager.cc:1347] Generated new cluster ID: ef8bede524004b85937964dccb3fb571
I20260928 06:43:02.805927 22630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:43:02.822096 22630 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:43:02.823360 22630 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:43:02.831908 22630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Generated new TSK 0
I20260928 06:43:02.832710 22630 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:43:02.868659 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45013
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:03.003238 22643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:03.003521 22643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:03.003587 22643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:03.007784 22643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:03.007850 22643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:03.007933 22643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:43:03.013254 22643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:03.014326 22643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:03.015403 22643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:03.024770 22649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:03.024771 22651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:03.024760 22648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:03.025187 22643 server_base.cc:1034] running on GCE node
I20260928 06:43:03.025563 22643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:03.026206 22643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:03.027379 22643 hybrid_clock.cc:648] HybridClock initialized: now 1790577783027360 us; error 30 us; skew 500 ppm
Sep 28 06:43:03 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577783, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:03.030167 22643 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:43:03.031100 22643 webserver.cc:460] Webserver started at http://127.8.151.193:40639/ using document root <none> and password file <none>
I20260928 06:43:03.031683 22643 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:03.031750 22643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:03.031944 22643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:43:03.033941 22643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "9ce87b5fa7d7411cb73410ef8a88e84d"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "0352cc2cb6f82591a4fefdb4401eaf4b"
server_key_iv: "0355c26f738d03cb35c8b95d07296ab4"
server_key_version: "encryptionkey@0"
I20260928 06:43:03.034467 22643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "9ce87b5fa7d7411cb73410ef8a88e84d"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "0352cc2cb6f82591a4fefdb4401eaf4b"
server_key_iv: "0355c26f738d03cb35c8b95d07296ab4"
server_key_version: "encryptionkey@0"
I20260928 06:43:03.038290 22643 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260928 06:43:03.040776 22658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:03.041998 22643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:43:03.042101 22643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "9ce87b5fa7d7411cb73410ef8a88e84d"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "0352cc2cb6f82591a4fefdb4401eaf4b"
server_key_iv: "0355c26f738d03cb35c8b95d07296ab4"
server_key_version: "encryptionkey@0"
I20260928 06:43:03.042196 22643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:03.067238 22643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:03.070254 22643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:03.070451 22643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:03.071076 22643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:03.071977 22643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:03.072026 22643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:03.072080 22643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:03.072116 22643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:03.081718 22643 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:40065
I20260928 06:43:03.081737 22771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:40065 every 8 connection(s)
I20260928 06:43:03.082646 22643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:43:03.086241 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 22643
I20260928 06:43:03.086372 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:43:03.086637 8799 external_mini_cluster.cc:1428] Setting key 2978e6069cd20fbb8ed4d79e6a348561
Sep 28 06:43:03 dist-test-slave-gtr8 krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577783, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:43:03.097203 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.084481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:45435 (local address 127.8.151.254:45013)
0928 06:43:03.084660 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:03.084665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:03.085580 (+ 915us) server_negotiation.cc:407] Connection header received
0928 06:43:03.087162 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:03.087165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:03.087220 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:03.087302 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:03.088800 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.089530 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.090669 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.090972 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.091073 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.093702 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:03.093705 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:03.093707 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:03.093731 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:03.095103 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:03.095656 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:03.095660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:03.095662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:03.095742 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:03.096117 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:03.096120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:03.096122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:03.096318 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:03.096401 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:03.096944 (+ 543us) server_negotiation.cc:299] Negotiation successful
0928 06:43:03.097069 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 06:43:03.098515 22774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.084895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:45435)
0928 06:43:03.085396 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:03.085456 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:03.086902 (+ 1446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:03.087502 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:03.087510 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:03.087982 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:03.088632 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.088652 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.089655 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.089659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.090529 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.090539 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.091133 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.091160 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.091351 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.091966 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:03.091988 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:03.093539 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:03.095288 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:03.095294 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:03.095309 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:03.095557 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:03.095862 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:03.095865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:03.095884 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:03.096024 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:03.096416 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:03.096423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:03.096676 (+ 253us) client_negotiation.cc:769] Sending connection context
0928 06:43:03.097365 (+ 689us) client_negotiation.cc:240] Negotiation successful
0928 06:43:03.097751 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:43:03.099921 22772 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45013
I20260928 06:43:03.100175 22772 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:03.100745 22772 heartbeater.cc:511] Master 127.8.151.254:45013 requested a full tablet report, sending...
I20260928 06:43:03.102289 22581 ts_manager.cc:194] Registered new tserver with Master: 9ce87b5fa7d7411cb73410ef8a88e84d (127.8.151.193:40065)
I20260928 06:43:03.103614 22581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:45435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:43:03.118115 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45013
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:03.236660 22777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:03.236915 22777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:03.236958 22777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:03.241016 22777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:03.241080 22777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:03.241146 22777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:43:03.246486 22777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:03.247575 22777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:03.248590 22777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:03.259155 22785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:03.259131 22783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:03.259132 22782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:03.259567 22777 server_base.cc:1034] running on GCE node
I20260928 06:43:03.259953 22777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:03.260576 22777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:03.261739 22777 hybrid_clock.cc:648] HybridClock initialized: now 1790577783261709 us; error 44 us; skew 500 ppm
Sep 28 06:43:03 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577783, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:03.264521 22777 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:43:03.265507 22777 webserver.cc:460] Webserver started at http://127.8.151.194:33213/ using document root <none> and password file <none>
I20260928 06:43:03.266142 22777 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:03.266220 22777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:03.266427 22777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:43:03.268904 22777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "99bcaaf1c9d9535663646842e196c368"
server_key_iv: "b8c11f7f83fe814c7979327579f24362"
server_key_version: "encryptionkey@0"
I20260928 06:43:03.269518 22777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "99bcaaf1c9d9535663646842e196c368"
server_key_iv: "b8c11f7f83fe814c7979327579f24362"
server_key_version: "encryptionkey@0"
I20260928 06:43:03.273890 22777 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:43:03.276733 22792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:03.278158 22777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 06:43:03.278277 22777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "99bcaaf1c9d9535663646842e196c368"
server_key_iv: "b8c11f7f83fe814c7979327579f24362"
server_key_version: "encryptionkey@0"
I20260928 06:43:03.278388 22777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:03.293234 22777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:03.296679 22777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:03.296851 22777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:03.297516 22777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:03.298441 22777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:03.298499 22777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:03.298547 22777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:03.298573 22777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:03.308264 22777 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:42333
I20260928 06:43:03.308303 22905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:42333 every 8 connection(s)
I20260928 06:43:03.309279 22777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:43:03.314992 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 22777
I20260928 06:43:03.315119 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:43:03.315409 8799 external_mini_cluster.cc:1428] Setting key b39680dbe3f3797c494e4268cbbce942
Sep 28 06:43:03 dist-test-slave-gtr8 krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577783, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:03.323729 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.311096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:57385 (local address 127.8.151.254:45013)
0928 06:43:03.311239 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:03.311243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:03.312258 (+ 1015us) server_negotiation.cc:407] Connection header received
0928 06:43:03.313497 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:03.313502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:03.313562 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:03.313645 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:03.314773 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.315405 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.316800 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.317122 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.317221 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.319846 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:03.319850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:03.319853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:03.319879 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:03.321395 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:03.322031 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:03.322034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:03.322036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:03.322126 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:03.322506 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:03.322509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:03.322511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:03.322722 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:03.322813 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:03.323448 (+ 635us) server_negotiation.cc:299] Negotiation successful
0928 06:43:03.323572 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 06:43:03.324841 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.311613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:57385)
0928 06:43:03.312091 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:03.312145 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:03.313297 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:03.313809 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:03.313816 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:03.314170 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:03.314629 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.314644 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.315543 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.315546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.316695 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.316707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.317287 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.317291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.317469 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.318104 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:03.318123 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:03.319689 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:03.321513 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:03.321519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:03.321535 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:03.321914 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:03.322218 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:03.322222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:03.322224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:03.322409 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:03.322827 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:03.322834 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:03.323182 (+ 348us) client_negotiation.cc:769] Sending connection context
0928 06:43:03.323404 (+ 222us) client_negotiation.cc:240] Negotiation successful
0928 06:43:03.323766 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
I20260928 06:43:03.325984 22906 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45013
I20260928 06:43:03.326215 22906 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:03.326715 22906 heartbeater.cc:511] Master 127.8.151.254:45013 requested a full tablet report, sending...
I20260928 06:43:03.327782 22581 ts_manager.cc:194] Registered new tserver with Master: bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
I20260928 06:43:03.328519 22581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:57385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:43:03.342288 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45013
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:03.452431 22911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:03.452695 22911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:03.452741 22911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:03.457022 22911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:03.457084 22911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:03.457159 22911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:43:03.462520 22911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:03.463657 22911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:03.464656 22911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:03.473979 22917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:03.474090 22919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:03.474260 22911 server_base.cc:1034] running on GCE node
W20260928 06:43:03.473981 22916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:03.474759 22911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:03.475407 22911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:03.476565 22911 hybrid_clock.cc:648] HybridClock initialized: now 1790577783476538 us; error 45 us; skew 500 ppm
Sep 28 06:43:03 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577783, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:03.479317 22911 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:43:03.480187 22911 webserver.cc:460] Webserver started at http://127.8.151.195:43251/ using document root <none> and password file <none>
I20260928 06:43:03.480720 22911 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:03.480784 22911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:03.480983 22911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:43:03.483031 22911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "f6a72efc7c91412da07b3f2ee51985b0"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "9e615b4fa71e0628ce9e339cb6bd8c0d"
server_key_iv: "6f2750087e36949fccfc53168c76b27c"
server_key_version: "encryptionkey@0"
I20260928 06:43:03.483609 22911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "f6a72efc7c91412da07b3f2ee51985b0"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "9e615b4fa71e0628ce9e339cb6bd8c0d"
server_key_iv: "6f2750087e36949fccfc53168c76b27c"
server_key_version: "encryptionkey@0"
I20260928 06:43:03.487303 22911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:43:03.489741 22926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:03.491009 22911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:43:03.491115 22911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "f6a72efc7c91412da07b3f2ee51985b0"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "9e615b4fa71e0628ce9e339cb6bd8c0d"
server_key_iv: "6f2750087e36949fccfc53168c76b27c"
server_key_version: "encryptionkey@0"
I20260928 06:43:03.491258 22911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:03.510926 22911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:03.514122 22911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:03.514297 22911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:03.514887 22911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:03.515733 22911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:03.515779 22911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:03.515831 22911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:03.515861 22911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:03.525480 22911 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:33793
I20260928 06:43:03.525498 23039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:33793 every 8 connection(s)
I20260928 06:43:03.526371 22911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:43:03.528455 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 22911
I20260928 06:43:03.528568 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:43:03.528810 8799 external_mini_cluster.cc:1428] Setting key b44b71658d342c02e4b419b69c97a627
Sep 28 06:43:03 dist-test-slave-gtr8 krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577783, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:03.539983 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.528211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:57475 (local address 127.8.151.254:45013)
0928 06:43:03.529211 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:03.529216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:03.529369 (+ 153us) server_negotiation.cc:407] Connection header received
0928 06:43:03.530880 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:03.530884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:03.530958 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:03.531028 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:03.532185 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.532709 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.533807 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.534030 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.534113 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.536640 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:03.536643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:03.536645 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:03.536669 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:03.537974 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:03.538503 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:03.538505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:03.538506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:03.538573 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:03.538904 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:03.538907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:03.538908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:03.539086 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:03.539225 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:03.539765 (+ 540us) server_negotiation.cc:299] Negotiation successful
0928 06:43:03.539862 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903}
I20260928 06:43:03.541047 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.528692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:57475)
0928 06:43:03.529210 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:03.529269 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:03.530692 (+ 1423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:03.531248 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:03.531256 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:03.531588 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:03.532049 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.532062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.532821 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.532825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.533697 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.533706 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.534164 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.534168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.534334 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.534860 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:03.534875 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:03.536483 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:03.538086 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:03.538092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:03.538106 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:03.538421 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:03.538644 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:03.538646 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:03.538648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:03.538811 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:03.539215 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:03.539221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:03.539511 (+ 290us) client_negotiation.cc:769] Sending connection context
0928 06:43:03.539707 (+ 196us) client_negotiation.cc:240] Negotiation successful
0928 06:43:03.540041 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":121,"threads_started":1}
I20260928 06:43:03.542151 23040 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45013
I20260928 06:43:03.542428 23040 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:03.542943 23040 heartbeater.cc:511] Master 127.8.151.254:45013 requested a full tablet report, sending...
I20260928 06:43:03.543955 22580 ts_manager.cc:194] Registered new tserver with Master: f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793)
I20260928 06:43:03.544662 22580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:57475
I20260928 06:43:03.554419 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:43:03.563305 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.556374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42530 (local address 127.8.151.254:45013)
0928 06:43:03.556502 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:03.556507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:03.556722 (+ 215us) server_negotiation.cc:407] Connection header received
0928 06:43:03.556833 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:03.556835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:03.556880 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:03.556942 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:03.557606 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.558113 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.558917 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.559184 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.559281 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.560580 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:03.560584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:03.560587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:03.560612 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:03.561753 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:03.562112 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:03.562116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:03.562118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:03.562207 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:03.562441 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:03.562445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:03.562446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:03.562659 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:03.562780 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:03.563001 (+ 221us) server_negotiation.cc:299] Negotiation successful
0928 06:43:03.563109 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 06:43:03.565979 22580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:43:03.569006 22580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:43:03.583770 23051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.577476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:40065 (local address 127.0.0.1:38902)
0928 06:43:03.577804 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:03.577849 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:03.577965 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:03.578907 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:03.578913 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:03.578935 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:03.579076 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.579083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.580641 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.580644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.582022 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.582031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.583172 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.583176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.583372 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.583417 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 06:43:03.583510 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 06:43:03.583618 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":37,"thread_start_us":83,"threads_started":1}
I20260928 06:43:03.584885 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.578164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38902 (local address 127.8.151.193:40065)
0928 06:43:03.578406 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:03.578422 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:03.578445 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:43:03.578521 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:03.578526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:03.578745 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:03.579261 (+ 516us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:03.579427 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.580504 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.582158 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.583032 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.584469 (+ 1437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.584609 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:03.584655 (+ 46us) server_negotiation.cc:299] Negotiation successful
0928 06:43:03.584746 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":67,"threads_started":1}
I20260928 06:43:03.585042 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.578544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:33793 (local address 127.0.0.1:49756)
0928 06:43:03.579817 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:03.579829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:03.579917 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:03.580485 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:03.580488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:03.580504 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:03.580598 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.580604 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.582133 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.582136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.583389 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.583400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.584667 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.584670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.584763 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.584784 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 06:43:03.584848 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 06:43:03.584909 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"mutex_wait_us":54,"thread_start_us":87,"threads_started":1}
I20260928 06:43:03.585438 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.578735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.8.151.195:33793)
0928 06:43:03.579065 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:03.579077 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:03.579925 (+ 848us) server_negotiation.cc:407] Connection header received
0928 06:43:03.580012 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:03.580017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:03.580198 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:03.580341 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:03.580720 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.581992 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.583515 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.584378 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.584472 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.584606 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:03.585220 (+ 614us) server_negotiation.cc:299] Negotiation successful
0928 06:43:03.585309 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":158,"threads_started":1}
I20260928 06:43:03.586184 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.578876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42333 (local address 127.0.0.1:46256)
0928 06:43:03.580060 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:03.580071 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:03.580132 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:03.580768 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:03.580770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:03.580787 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:03.580868 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.580874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.582945 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.582948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.584027 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.584038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.585853 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.585856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.585940 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.585962 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 06:43:03.586008 (+ 46us) client_negotiation.cc:240] Negotiation successful
0928 06:43:03.586068 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1136,"thread_start_us":89,"threads_started":1}
I20260928 06:43:03.586870 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.579404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46256 (local address 127.8.151.194:42333)
0928 06:43:03.580350 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:03.580360 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:03.580380 (+ 20us) server_negotiation.cc:407] Connection header received
0928 06:43:03.580445 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:03.580450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:03.580622 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:03.580999 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:03.581154 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.582802 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.584182 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.585730 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.586471 (+ 741us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.586600 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:03.586645 (+ 45us) server_negotiation.cc:299] Negotiation successful
0928 06:43:03.586738 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":855,"thread_start_us":121,"threads_started":1}
I20260928 06:43:03.588162 22974 tablet_service.cc:1467] Processing CreateTablet for tablet 16ae14898d7c435993bf9782e7833718 (DEFAULT_TABLE table=test-table [id=bf1c8e686e6d4b7fa25db83f709a4184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:03.588181 22706 tablet_service.cc:1467] Processing CreateTablet for tablet 16ae14898d7c435993bf9782e7833718 (DEFAULT_TABLE table=test-table [id=bf1c8e686e6d4b7fa25db83f709a4184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:03.589231 22706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16ae14898d7c435993bf9782e7833718. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:03.589233 22974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16ae14898d7c435993bf9782e7833718. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:03.589330 22840 tablet_service.cc:1467] Processing CreateTablet for tablet 16ae14898d7c435993bf9782e7833718 (DEFAULT_TABLE table=test-table [id=bf1c8e686e6d4b7fa25db83f709a4184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:03.590204 22840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16ae14898d7c435993bf9782e7833718. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:03.597738 23059 tablet_bootstrap.cc:492] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: Bootstrap starting.
I20260928 06:43:03.597738 23058 tablet_bootstrap.cc:492] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Bootstrap starting.
I20260928 06:43:03.598654 23057 tablet_bootstrap.cc:492] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: Bootstrap starting.
I20260928 06:43:03.600090 23058 tablet_bootstrap.cc:654] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:03.600478 23059 tablet_bootstrap.cc:654] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:03.601042 23058 log.cc:824] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:03.601301 23059 log.cc:824] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:03.601858 23057 tablet_bootstrap.cc:654] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:03.602713 23057 log.cc:824] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:03.603400 23058 tablet_bootstrap.cc:492] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: No bootstrap required, opened a new log
I20260928 06:43:03.603657 23058 ts_tablet_manager.cc:1397] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:43:03.603703 23059 tablet_bootstrap.cc:492] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: No bootstrap required, opened a new log
I20260928 06:43:03.603900 23059 ts_tablet_manager.cc:1397] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:43:03.604682 23057 tablet_bootstrap.cc:492] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: No bootstrap required, opened a new log
I20260928 06:43:03.604867 23057 ts_tablet_manager.cc:1397] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:43:03.607808 23058 raft_consensus.cc:348] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:03.607807 23059 raft_consensus.cc:348] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:03.608117 23058 raft_consensus.cc:374] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:03.608203 23058 raft_consensus.cc:729] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ce87b5fa7d7411cb73410ef8a88e84d, State: Initialized, Role: FOLLOWER
I20260928 06:43:03.608199 23059 raft_consensus.cc:374] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:03.608461 23059 raft_consensus.cc:729] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd8fbe08cc1408987b0b2c0efebf0f2, State: Initialized, Role: FOLLOWER
I20260928 06:43:03.608383 23057 raft_consensus.cc:348] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:03.608707 23058 consensus_queue.cc:260] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:03.609030 23059 consensus_queue.cc:260] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:03.609364 22772 heartbeater.cc:503] Master 127.8.151.254:45013 was elected leader, sending a full tablet report...
I20260928 06:43:03.609706 23057 raft_consensus.cc:374] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:03.609773 23057 raft_consensus.cc:729] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a72efc7c91412da07b3f2ee51985b0, State: Initialized, Role: FOLLOWER
I20260928 06:43:03.610183 23057 consensus_queue.cc:260] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:03.610673 23058 ts_tablet_manager.cc:1428] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260928 06:43:03.611064 23057 ts_tablet_manager.cc:1428] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260928 06:43:03.611430 23040 heartbeater.cc:503] Master 127.8.151.254:45013 was elected leader, sending a full tablet report...
I20260928 06:43:03.611474 23059 ts_tablet_manager.cc:1428] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260928 06:43:03.611778 22906 heartbeater.cc:503] Master 127.8.151.254:45013 was elected leader, sending a full tablet report...
W20260928 06:43:03.777377 23041 tablet.cc:2369] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 06:43:03.810504 22907 tablet.cc:2369] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 06:43:03.834276 22773 tablet.cc:2369] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:43:03.868877 23064 raft_consensus.cc:482] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:43:03.869114 23064 raft_consensus.cc:504] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:03.870311 23064 leader_election.cc:288] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ce87b5fa7d7411cb73410ef8a88e84d (127.8.151.193:40065), f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793)
I20260928 06:43:03.874467 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.870572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:33793 (local address 127.8.151.194:51701)
0928 06:43:03.870978 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:03.870993 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:03.871102 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:03.871360 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:03.871362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:03.871379 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:03.871520 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.871528 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.872422 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.872425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.873313 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.873322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.874034 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.874038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.874136 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.874161 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 06:43:03.874234 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 06:43:03.874301 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":20,"spinlock_wait_cycles":17536,"thread_start_us":77,"threads_started":1}
I20260928 06:43:03.874464 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.870896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:51701 (local address 127.8.151.195:33793)
0928 06:43:03.871025 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:03.871029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:03.871047 (+ 18us) server_negotiation.cc:407] Connection header received
0928 06:43:03.871222 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:03.871225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:03.871271 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:03.871330 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:03.871642 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.872305 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.873433 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.873917 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.874041 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.874097 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:03.874256 (+ 159us) server_negotiation.cc:299] Negotiation successful
0928 06:43:03.874337 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:43:03.874925 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.870714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:53257 (local address 127.8.151.193:40065)
0928 06:43:03.870844 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:03.870848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:03.871351 (+ 503us) server_negotiation.cc:407] Connection header received
0928 06:43:03.871565 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:03.871569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:03.871624 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:03.871694 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:03.871926 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.872447 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.873723 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.874280 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.874383 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.874430 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:03.874746 (+ 316us) server_negotiation.cc:299] Negotiation successful
0928 06:43:03.874822 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 06:43:03.874934 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.870575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:40065 (local address 127.8.151.194:53257)
0928 06:43:03.871258 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:03.871272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:03.871447 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:03.871710 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:03.871712 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:03.871727 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:03.871830 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.871835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.872555 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.872559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.873595 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:03.873606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.874502 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.874505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:03.874599 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.874622 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:43:03.874723 (+ 101us) client_negotiation.cc:240] Negotiation successful
0928 06:43:03.874791 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":134,"spinlock_wait_cycles":896,"thread_start_us":64,"threads_started":1}
I20260928 06:43:03.875517 22994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16ae14898d7c435993bf9782e7833718" candidate_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a72efc7c91412da07b3f2ee51985b0" is_pre_election: true
I20260928 06:43:03.875523 22726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16ae14898d7c435993bf9782e7833718" candidate_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" is_pre_election: true
I20260928 06:43:03.875846 22994 raft_consensus.cc:2454] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdd8fbe08cc1408987b0b2c0efebf0f2 in term 0.
I20260928 06:43:03.875846 22726 raft_consensus.cc:2454] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdd8fbe08cc1408987b0b2c0efebf0f2 in term 0.
I20260928 06:43:03.876327 22796 leader_election.cc:302] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ce87b5fa7d7411cb73410ef8a88e84d, bdd8fbe08cc1408987b0b2c0efebf0f2; no voters:
I20260928 06:43:03.876657 23064 raft_consensus.cc:2781] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:43:03.876763 23064 raft_consensus.cc:482] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:43:03.876803 23064 raft_consensus.cc:3037] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:03.878053 23064 raft_consensus.cc:504] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:03.878450 23064 leader_election.cc:288] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9ce87b5fa7d7411cb73410ef8a88e84d (127.8.151.193:40065), f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793)
I20260928 06:43:03.878799 22994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16ae14898d7c435993bf9782e7833718" candidate_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a72efc7c91412da07b3f2ee51985b0"
I20260928 06:43:03.878849 22726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16ae14898d7c435993bf9782e7833718" candidate_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d"
I20260928 06:43:03.878923 22994 raft_consensus.cc:3037] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:03.878976 22726 raft_consensus.cc:3037] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:03.880043 22994 raft_consensus.cc:2454] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdd8fbe08cc1408987b0b2c0efebf0f2 in term 1.
I20260928 06:43:03.880218 22726 raft_consensus.cc:2454] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdd8fbe08cc1408987b0b2c0efebf0f2 in term 1.
I20260928 06:43:03.880344 22795 leader_election.cc:302] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdd8fbe08cc1408987b0b2c0efebf0f2, f6a72efc7c91412da07b3f2ee51985b0; no voters:
I20260928 06:43:03.880522 23064 raft_consensus.cc:2781] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:43:03.880792 23064 raft_consensus.cc:686] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 1 LEADER]: Becoming Leader. State: Replica: bdd8fbe08cc1408987b0b2c0efebf0f2, State: Running, Role: LEADER
I20260928 06:43:03.881161 23064 consensus_queue.cc:237] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:03.884845 22581 catalog_manager.cc:5495] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194). New cstate: current_term: 1 leader_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } health_report { overall_health: UNKNOWN } } }
I20260928 06:43:03.954818 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.951484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46268 (local address 127.8.151.194:42333)
0928 06:43:03.951624 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:03.951629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:03.951652 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:43:03.951737 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:03.951740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:03.951795 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:03.951884 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:43:03.952115 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.952613 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.953432 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:03.953662 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:03.953745 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:03.954036 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:43:03.954448 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:43:03.954577 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:03.954610 (+ 33us) server_negotiation.cc:299] Negotiation successful
0928 06:43:03.954680 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:43:03.961129 22726 raft_consensus.cc:1261] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Refusing update from remote peer bdd8fbe08cc1408987b0b2c0efebf0f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:43:03.961129 22994 raft_consensus.cc:1261] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0 [term 1 FOLLOWER]: Refusing update from remote peer bdd8fbe08cc1408987b0b2c0efebf0f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:43:03.961850 23064 consensus_queue.cc:1035] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:43:03.962037 23068 consensus_queue.cc:1035] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:43:03.968768 23071 mvcc.cc:205] Tried to move back new op lower bound from 7334206603097559040 to 7334206602780217344. Current Snapshot: MvccSnapshot[applied={T|T < 7334206603097559040}]
I20260928 06:43:03.973584 23073 mvcc.cc:205] Tried to move back new op lower bound from 7334206603097559040 to 7334206602780217344. Current Snapshot: MvccSnapshot[applied={T|T < 7334206603097559040}]
I20260928 06:43:03.978410 22581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:43:03.978590 22581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:43:03.981177 22581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Sending DeleteTablet for 3 replicas of tablet 16ae14898d7c435993bf9782e7833718
I20260928 06:43:03.982122 22974 tablet_service.cc:1514] Processing DeleteTablet for tablet 16ae14898d7c435993bf9782e7833718 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:03 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:49756
I20260928 06:43:03.982388 22840 tablet_service.cc:1514] Processing DeleteTablet for tablet 16ae14898d7c435993bf9782e7833718 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:03 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:46256
I20260928 06:43:03.983016 23082 tablet_replica.cc:330] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: stopping tablet replica
I20260928 06:43:03.983065 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 22643
I20260928 06:43:03.983381 23082 raft_consensus.cc:2229] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:43:03.983770 23082 raft_consensus.cc:2258] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:43:03.985406 23082 ts_tablet_manager.cc:1905] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:43:03.988153 23082 ts_tablet_manager.cc:1918] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:43:03.988253 23082 log.cc:1196] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/16ae14898d7c435993bf9782e7833718
I20260928 06:43:03.988495 23081 tablet_replica.cc:330] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: stopping tablet replica
I20260928 06:43:03.988817 23081 raft_consensus.cc:2229] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:43:03.989190 23081 raft_consensus.cc:2258] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:43:03.989681 23082 ts_tablet_manager.cc:1939] T 16ae14898d7c435993bf9782e7833718 P bdd8fbe08cc1408987b0b2c0efebf0f2: Deleting consensus metadata
I20260928 06:43:03.990690 23081 ts_tablet_manager.cc:1905] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:43:03.990897 22565 catalog_manager.cc:4841] TS bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333): tablet 16ae14898d7c435993bf9782e7833718 (table test-table [id=bf1c8e686e6d4b7fa25db83f709a4184]) successfully deleted
W20260928 06:43:03.991768 22568 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:40065 (error 108)
I20260928 06:43:03.992733 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:03.992465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:40065 (local address 127.0.0.1:38906)
0928 06:43:03.992581 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:03.992651 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:40065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 06:43:03.992894 22568 catalog_manager.cc:4584] TS 9ce87b5fa7d7411cb73410ef8a88e84d (127.8.151.193:40065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16ae14898d7c435993bf9782e7833718: Network error: Client connection negotiation failed: client connection to 127.8.151.193:40065: connect: Connection refused (error 111)
I20260928 06:43:03.994674 23081 ts_tablet_manager.cc:1918] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:43:03.994767 23081 log.cc:1196] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/16ae14898d7c435993bf9782e7833718
I20260928 06:43:03.995070 23081 ts_tablet_manager.cc:1939] T 16ae14898d7c435993bf9782e7833718 P f6a72efc7c91412da07b3f2ee51985b0: Deleting consensus metadata
I20260928 06:43:03.996362 22567 catalog_manager.cc:4841] TS f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793): tablet 16ae14898d7c435993bf9782e7833718 (table test-table [id=bf1c8e686e6d4b7fa25db83f709a4184]) successfully deleted
I20260928 06:43:03.999750 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 22777
I20260928 06:43:04.005432 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 22911
I20260928 06:43:04.010224 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 22549
I20260928 06:43:04.015574 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:45013
--webserver_interface=127.8.151.254
--webserver_port=42739
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:45013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:04.123829 23084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:04.124111 23084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:04.124151 23084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:04.128259 23084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:43:04.128326 23084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:04.128345 23084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:04.128361 23084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:43:04.128376 23084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:43:04.133785 23084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:45013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:45013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=42739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:04.134886 23084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:04.135986 23084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:04.150501 23091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:04.150511 23090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:04.150542 23093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:04.150843 23084 server_base.cc:1034] running on GCE node
I20260928 06:43:04.151504 23084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:04.152717 23084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:04.153908 23084 hybrid_clock.cc:648] HybridClock initialized: now 1790577784153877 us; error 49 us; skew 500 ppm
Sep 28 06:43:04 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577784, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:04.156868 23084 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:43:04.157836 23084 webserver.cc:460] Webserver started at http://127.8.151.254:42739/ using document root <none> and password file <none>
I20260928 06:43:04.158409 23084 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:04.158484 23084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:04.162760 23084 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:43:04.165123 23100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:04.166579 23084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:43:04.166726 23084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "90d8dfb9ad58457c94e9c1fbd0132052"
format_stamp: "Formatted at 2026-09-28 06:43:02 on dist-test-slave-gtr8"
server_key: "797658db8a037efbc34b64bd3a09d745"
server_key_iv: "b1ac740d255853e95edd9b8aaa913cfb"
server_key_version: "encryptionkey@0"
I20260928 06:43:04.167382 23084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:04.183911 23084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:04.187050 23084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:04.187263 23084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:04.194770 23084 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:45013
I20260928 06:43:04.195041 23152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:45013 every 8 connection(s)
I20260928 06:43:04.196013 23084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:43:04.200544 23153 sys_catalog.cc:263] Verifying existing consensus state
I20260928 06:43:04.201651 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 23084
I20260928 06:43:04.202158 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:40065
--local_ip_for_outbound_sockets=127.8.151.193
--tserver_master_addrs=127.8.151.254:45013
--webserver_port=40639
--webserver_interface=127.8.151.193
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:43:04.202407 23153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Bootstrap starting.
I20260928 06:43:04.225454 23153 log.cc:824] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:04.236732 23153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:43:04.237169 23153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Bootstrap complete.
I20260928 06:43:04.242491 23153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:04.242992 23153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d8dfb9ad58457c94e9c1fbd0132052, State: Initialized, Role: FOLLOWER
I20260928 06:43:04.243579 23153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:04.243736 23153 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:43:04.243821 23153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:43:04.243918 23153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 1 FOLLOWER]: Advancing to term 2
I20260928 06:43:04.246519 23153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:04.246893 23153 leader_election.cc:302] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90d8dfb9ad58457c94e9c1fbd0132052; no voters:
I20260928 06:43:04.247305 23153 leader_election.cc:288] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 06:43:04.247517 23157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 06:43:04.247797 23157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 2 LEADER]: Becoming Leader. State: Replica: 90d8dfb9ad58457c94e9c1fbd0132052, State: Running, Role: LEADER
I20260928 06:43:04.248169 23157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:04.248660 23153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:43:04.249261 23157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90d8dfb9ad58457c94e9c1fbd0132052. Latest consensus state: current_term: 2 leader_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } } }
I20260928 06:43:04.249475 23157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:04.249642 23158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } } }
I20260928 06:43:04.249732 23158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:04.254155 23172 catalog_manager.cc:1259] Loaded cluster ID: ef8bede524004b85937964dccb3fb571
I20260928 06:43:04.254220 23172 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: loading cluster ID for follower catalog manager: success
W20260928 06:43:04.255546 23172 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260928 06:43:04.255617 23172 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260928 06:43:04.255687 23166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:43:04.258025 23166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:43:04.258471 23166 catalog_manager.cc:1259] Loaded cluster ID: ef8bede524004b85937964dccb3fb571
I20260928 06:43:04.258527 23166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:43:04.259517 23166 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260928 06:43:04.263916 23166 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260928 06:43:04.269284 23166 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:43:04.270032 23166 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Loaded TSK: 0
I20260928 06:43:04.271359 23166 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 06:43:04.335117 23155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:04.335443 23155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:04.335506 23155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:04.339957 23155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:04.340030 23155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:04.340116 23155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:43:04.348237 23155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:40065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=40639
--tserver_master_addrs=127.8.151.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:04.349874 23155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:04.351462 23155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:43:04.365038 23155 server_base.cc:1034] running on GCE node
W20260928 06:43:04.365213 23181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:04.365450 23180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:04.365865 23183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:04.366361 23155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:04.367209 23155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:04.368394 23155 hybrid_clock.cc:648] HybridClock initialized: now 1790577784368366 us; error 41 us; skew 500 ppm
Sep 28 06:43:04 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577784, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:04.371937 23155 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:43:04.373092 23155 webserver.cc:460] Webserver started at http://127.8.151.193:40639/ using document root <none> and password file <none>
I20260928 06:43:04.373893 23155 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:04.374058 23155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:04.379474 23155 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:43:04.382278 23190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:04.383903 23155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:43:04.384073 23155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "9ce87b5fa7d7411cb73410ef8a88e84d"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "0352cc2cb6f82591a4fefdb4401eaf4b"
server_key_iv: "0355c26f738d03cb35c8b95d07296ab4"
server_key_version: "encryptionkey@0"
I20260928 06:43:04.384919 23155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:04.415282 23155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:04.419096 23155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:04.419366 23155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:04.420105 23155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:04.421427 23197 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 06:43:04.425871 23155 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 06:43:04.425923 23155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260928 06:43:04.425976 23155 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 06:43:04.428037 23155 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 06:43:04.428092 23155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260928 06:43:04.428421 23197 tablet_bootstrap.cc:492] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Bootstrap starting.
I20260928 06:43:04.437516 23155 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:40065
I20260928 06:43:04.437572 23304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:40065 every 8 connection(s)
I20260928 06:43:04.438589 23155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:43:04.440560 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 23155
I20260928 06:43:04.441083 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:42333
--local_ip_for_outbound_sockets=127.8.151.194
--tserver_master_addrs=127.8.151.254:45013
--webserver_port=33213
--webserver_interface=127.8.151.194
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:43:04.448490 23197 log.cc:824] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Log is configured to *not* fsync() on all Append() calls
Sep 28 06:43:04 dist-test-slave-gtr8 krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577784, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:04.460862 23197 tablet_bootstrap.cc:492] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:43:04.461279 23197 tablet_bootstrap.cc:492] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Bootstrap complete.
I20260928 06:43:04.462628 23197 ts_tablet_manager.cc:1397] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Time spent bootstrapping tablet: real 0.034s user 0.022s sys 0.005s
I20260928 06:43:04.464074 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.443919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:34907 (local address 127.8.151.254:45013)
0928 06:43:04.444335 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:04.444354 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:04.444970 (+ 616us) server_negotiation.cc:407] Connection header received
0928 06:43:04.446300 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:04.446317 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:04.446734 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:04.447103 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:04.448968 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.450457 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.452264 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.452954 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.453113 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.457584 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:04.457591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:04.457608 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:04.457653 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:04.459622 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:04.460243 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:04.460249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:04.460255 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:04.460397 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:04.460795 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:04.460799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:04.460800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:04.461408 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:04.461529 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:04.462900 (+ 1371us) server_negotiation.cc:299] Negotiation successful
0928 06:43:04.463214 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":136,"threads_started":1}
I20260928 06:43:04.465337 23197 raft_consensus.cc:348] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:04.465723 23197 raft_consensus.cc:729] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ce87b5fa7d7411cb73410ef8a88e84d, State: Initialized, Role: FOLLOWER
I20260928 06:43:04.466176 23197 consensus_queue.cc:260] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } } peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } }
I20260928 06:43:04.467043 23197 ts_tablet_manager.cc:1428] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260928 06:43:04.467875 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.442494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:34907)
0928 06:43:04.444773 (+ 2279us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:04.444827 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:04.445796 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:04.447601 (+ 1805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:04.447610 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:04.448080 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:04.448771 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.448793 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.450651 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.450714 (+ 63us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.452117 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.452129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.453312 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.453316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.453496 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.455480 (+ 1984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:04.455501 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:04.457359 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:04.459791 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:04.459796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:04.459808 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:04.460114 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:04.460501 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:04.460504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:04.460506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:04.460688 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:04.461580 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:04.461586 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:04.461854 (+ 268us) client_negotiation.cc:769] Sending connection context
0928 06:43:04.462014 (+ 160us) client_negotiation.cc:240] Negotiation successful
0928 06:43:04.462316 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2072,"thread_start_us":176,"threads_started":1}
I20260928 06:43:04.470121 23305 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45013
I20260928 06:43:04.470340 23305 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:04.470824 23305 heartbeater.cc:511] Master 127.8.151.254:45013 requested a full tablet report, sending...
I20260928 06:43:04.473110 23117 ts_manager.cc:194] Registered new tserver with Master: 9ce87b5fa7d7411cb73410ef8a88e84d (127.8.151.193:40065)
I20260928 06:43:04.476475 23117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:34907
I20260928 06:43:04.480788 23305 heartbeater.cc:503] Master 127.8.151.254:45013 was elected leader, sending a full tablet report...
Sep 28 06:43:04 dist-test-slave-gtr8 krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577784, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:43:04.487378 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.475843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:40065 (local address 127.0.0.1:38912)
0928 06:43:04.476256 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:04.476307 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:04.476440 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:04.477401 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:04.477407 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:04.477836 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:04.477971 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.477979 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.479471 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.479475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.480394 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.480406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.481176 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.481180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.481289 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.481883 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:04.481898 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:04.483431 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:04.485498 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:04.485508 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:04.485514 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:04.485812 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:04.486126 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:04.486129 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:04.486131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:04.486225 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:04.486718 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:04.486724 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:04.486848 (+ 124us) client_negotiation.cc:769] Sending connection context
0928 06:43:04.486985 (+ 137us) client_negotiation.cc:240] Negotiation successful
0928 06:43:04.487204 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":116,"threads_started":1}
I20260928 06:43:04.487443 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.476620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.8.151.193:40065)
0928 06:43:04.476926 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:04.476937 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:04.476960 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:43:04.477029 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:04.477034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:04.477231 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:04.478118 (+ 887us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:04.478244 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.479341 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.480551 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.481038 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.481579 (+ 541us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.483588 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:04.483598 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:04.483611 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:04.483636 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:04.485377 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:04.485928 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:04.485934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:04.485936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:04.486036 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:04.486307 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:04.486310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:04.486311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:04.486609 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:04.486731 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:04.487078 (+ 347us) server_negotiation.cc:299] Negotiation successful
0928 06:43:04.487265 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":119,"threads_started":1}
I20260928 06:43:04.488879 23225 tablet_service.cc:1514] Processing DeleteTablet for tablet 16ae14898d7c435993bf9782e7833718 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:03 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:38912
I20260928 06:43:04.489886 23314 tablet_replica.cc:330] stopping tablet replica
I20260928 06:43:04.490283 23314 raft_consensus.cc:2229] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:43:04.490597 23314 raft_consensus.cc:2258] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:43:04.491889 23314 ts_tablet_manager.cc:1905] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:43:04.496706 23314 ts_tablet_manager.cc:1918] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:43:04.496876 23314 log.cc:1196] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/16ae14898d7c435993bf9782e7833718
I20260928 06:43:04.497234 23314 ts_tablet_manager.cc:1939] T 16ae14898d7c435993bf9782e7833718 P 9ce87b5fa7d7411cb73410ef8a88e84d: Deleting consensus metadata
I20260928 06:43:04.499356 23104 catalog_manager.cc:4841] TS 9ce87b5fa7d7411cb73410ef8a88e84d (127.8.151.193:40065): tablet 16ae14898d7c435993bf9782e7833718 (table test-table [id=bf1c8e686e6d4b7fa25db83f709a4184]) successfully deleted
W20260928 06:43:04.570005 23307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:04.570261 23307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:04.570319 23307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:04.574520 23307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:04.574589 23307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:04.574663 23307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:43:04.579871 23307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:42333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=33213
--tserver_master_addrs=127.8.151.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:04.580890 23307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:04.581867 23307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:04.592058 23319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:04.592065 23320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:04.592290 23307 server_base.cc:1034] running on GCE node
W20260928 06:43:04.592450 23322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:04.592839 23307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:04.593474 23307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:04.594648 23307 hybrid_clock.cc:648] HybridClock initialized: now 1790577784594622 us; error 36 us; skew 500 ppm
Sep 28 06:43:04 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577784, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:04.597570 23307 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:43:04.598470 23307 webserver.cc:460] Webserver started at http://127.8.151.194:33213/ using document root <none> and password file <none>
I20260928 06:43:04.598991 23307 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:04.599062 23307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:04.603271 23307 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260928 06:43:04.605216 23329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:04.606429 23307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:43:04.606523 23307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "99bcaaf1c9d9535663646842e196c368"
server_key_iv: "b8c11f7f83fe814c7979327579f24362"
server_key_version: "encryptionkey@0"
I20260928 06:43:04.607051 23307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:04.623404 23307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:04.626462 23307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:04.626629 23307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:04.627264 23307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:04.628072 23307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:04.628121 23307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:04.628165 23307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:04.628192 23307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:04.638051 23307 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:42333
I20260928 06:43:04.638065 23442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:42333 every 8 connection(s)
I20260928 06:43:04.639089 23307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:43:04.640792 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 23307
I20260928 06:43:04.641250 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:33793
--local_ip_for_outbound_sockets=127.8.151.195
--tserver_master_addrs=127.8.151.254:45013
--webserver_port=43251
--webserver_interface=127.8.151.195
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 06:43:04 dist-test-slave-gtr8 krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577784, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:04.652944 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.640886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:58453 (local address 127.8.151.254:45013)
0928 06:43:04.641017 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:04.641021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:04.641987 (+ 966us) server_negotiation.cc:407] Connection header received
0928 06:43:04.643581 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:04.643585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:04.643652 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:04.643751 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:04.645045 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.645576 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.646737 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.646974 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.647062 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.649534 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:04.649537 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:04.649540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:04.649567 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:04.650946 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:04.651470 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:04.651474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:04.651476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:04.651557 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:04.651878 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:04.651881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:04.651883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:04.652057 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:04.652137 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:04.652644 (+ 507us) server_negotiation.cc:299] Negotiation successful
0928 06:43:04.652776 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:43:04.653978 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.641335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:58453)
0928 06:43:04.641823 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:04.641876 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:04.643314 (+ 1438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:04.643961 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:04.643971 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:04.644381 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:04.644880 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.644894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.645732 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.645736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.646615 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.646624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.647125 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.647130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.647312 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.647848 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:04.647863 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:04.649369 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:04.651052 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:04.651057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:04.651072 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:04.651381 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:04.651644 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:04.651647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:04.651649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:04.651785 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:04.652160 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:04.652167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:04.652405 (+ 238us) client_negotiation.cc:769] Sending connection context
0928 06:43:04.652565 (+ 160us) client_negotiation.cc:240] Negotiation successful
0928 06:43:04.652893 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":121,"threads_started":1}
I20260928 06:43:04.655100 23443 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45013
I20260928 06:43:04.655377 23443 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:04.655936 23443 heartbeater.cc:511] Master 127.8.151.254:45013 requested a full tablet report, sending...
I20260928 06:43:04.656903 23117 ts_manager.cc:194] Registered new tserver with Master: bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333)
I20260928 06:43:04.657454 23117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:58453
W20260928 06:43:04.770584 23446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:04.770824 23446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:04.770865 23446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:04.774839 23446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:04.774895 23446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:04.774956 23446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:43:04.780157 23446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:33793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=43251
--tserver_master_addrs=127.8.151.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:04.781154 23446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:04.782137 23446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:04.791342 23454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:04.791471 23452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:04.791361 23451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:04.791671 23446 server_base.cc:1034] running on GCE node
I20260928 06:43:04.792107 23446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:04.792727 23446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:04.793905 23446 hybrid_clock.cc:648] HybridClock initialized: now 1790577784793889 us; error 27 us; skew 500 ppm
Sep 28 06:43:04 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577784, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:04.796635 23446 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:43:04.797564 23446 webserver.cc:460] Webserver started at http://127.8.151.195:43251/ using document root <none> and password file <none>
I20260928 06:43:04.798089 23446 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:04.798177 23446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:04.802071 23446 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:43:04.804141 23461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:04.805399 23446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:43:04.805497 23446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "f6a72efc7c91412da07b3f2ee51985b0"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "9e615b4fa71e0628ce9e339cb6bd8c0d"
server_key_iv: "6f2750087e36949fccfc53168c76b27c"
server_key_version: "encryptionkey@0"
I20260928 06:43:04.806031 23446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:04.816676 23446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:04.819871 23446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:04.820060 23446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:04.820785 23446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:04.821703 23446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:04.821758 23446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:04.821812 23446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:04.821839 23446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:04.831629 23446 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:33793
I20260928 06:43:04.831674 23574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:33793 every 8 connection(s)
I20260928 06:43:04.832794 23446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:43:04.838668 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 23446
Sep 28 06:43:04 dist-test-slave-gtr8 krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577784, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:04.847008 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.834643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:51601 (local address 127.8.151.254:45013)
0928 06:43:04.834772 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:04.834776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:04.835799 (+ 1023us) server_negotiation.cc:407] Connection header received
0928 06:43:04.837056 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:04.837059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:04.837121 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:04.837195 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:04.838361 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.839078 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.840499 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.840725 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.840809 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.843245 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:04.843249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:04.843251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:04.843285 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:04.844638 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:04.845218 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:04.845221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:04.845223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:04.845302 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:04.845841 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:04.845845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:04.845847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:04.846068 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:04.846189 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:04.846730 (+ 541us) server_negotiation.cc:299] Negotiation successful
0928 06:43:04.846843 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:43:04.847939 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.839353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.8.151.254:45013)
0928 06:43:04.839572 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:04.839576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:04.839829 (+ 253us) server_negotiation.cc:407] Connection header received
0928 06:43:04.839940 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:04.839943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:04.839988 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:04.840073 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:04.840889 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.841650 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.842737 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.843068 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.843214 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.844734 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:04.844738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:04.844740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:04.844765 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:04.846287 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:04.846731 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:04.846735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:04.846736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:04.846817 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:04.847098 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:04.847101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:04.847103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:04.847307 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:04.847500 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:04.847677 (+ 177us) server_negotiation.cc:299] Negotiation successful
0928 06:43:04.847786 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"spinlock_wait_cycles":8960,"thread_start_us":61,"threads_started":1}
I20260928 06:43:04.848513 23577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.835170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:51601)
0928 06:43:04.835638 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:04.835690 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:04.836840 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:04.837381 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:04.837388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:04.837734 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:04.838213 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.838226 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.839227 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.839231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.840392 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.840400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.840859 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.840863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.841026 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.841558 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:04.841574 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:04.843051 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:04.844763 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:04.844769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:04.844785 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:04.845089 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:04.845382 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:04.845385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:04.845387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:04.845561 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:04.846190 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:04.846198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:04.846497 (+ 299us) client_negotiation.cc:769] Sending connection context
0928 06:43:04.846663 (+ 166us) client_negotiation.cc:240] Negotiation successful
0928 06:43:04.846987 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":131,"threads_started":1}
I20260928 06:43:04.849607 23575 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45013
I20260928 06:43:04.849826 23575 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:04.850286 23575 heartbeater.cc:511] Master 127.8.151.254:45013 requested a full tablet report, sending...
I20260928 06:43:04.851290 23117 ts_manager.cc:194] Registered new tserver with Master: f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793)
I20260928 06:43:04.852035 23117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:51601
I20260928 06:43:04.870976 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:43:04.878897 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.871630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42560 (local address 127.8.151.254:45013)
0928 06:43:04.875271 (+ 3641us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:04.875276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:04.875501 (+ 225us) server_negotiation.cc:407] Connection header received
0928 06:43:04.875677 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:04.875680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:04.875787 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:04.875879 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:43:04.876135 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.876674 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.877513 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.877743 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.877830 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.878124 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:43:04.878517 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:43:04.878620 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:04.878691 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 06:43:04.878750 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3535}
I20260928 06:43:04.879667 23117 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:43:04.882910 23117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:43:04.890709 23225 tablet_service.cc:1467] Processing CreateTablet for tablet c71e834c54f1468894ace6e9f0f26f06 (DEFAULT_TABLE table=test-table [id=4fc62e1dd53c4749b3110fea8075820b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:04.891372 23225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c71e834c54f1468894ace6e9f0f26f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:04.894642 23197 tablet_bootstrap.cc:492] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Bootstrap starting.
I20260928 06:43:04.895493 23197 tablet_bootstrap.cc:654] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:04.896404 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.889940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:33793 (local address 127.0.0.1:49768)
0928 06:43:04.890179 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:04.890193 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:04.890283 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:04.891503 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:04.891505 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:04.891522 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:04.891615 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.891621 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.892972 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.892975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.894170 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.894181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.895921 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.895925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.896072 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.896121 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 06:43:04.896193 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 06:43:04.896268 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260928 06:43:04.896883 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.890335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.8.151.195:33793)
0928 06:43:04.890782 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:04.890794 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:04.890819 (+ 25us) server_negotiation.cc:407] Connection header received
0928 06:43:04.890898 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:04.890904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:04.891170 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:04.891329 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:04.891741 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.892825 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.894298 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.895565 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.895672 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.895846 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:04.896645 (+ 799us) server_negotiation.cc:299] Negotiation successful
0928 06:43:04.896745 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":154,"threads_started":1}
I20260928 06:43:04.897070 23197 tablet_bootstrap.cc:492] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: No bootstrap required, opened a new log
I20260928 06:43:04.897145 23197 ts_tablet_manager.cc:1397] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 06:43:04.897535 23197 raft_consensus.cc:348] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:04.897660 23197 raft_consensus.cc:374] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:04.897698 23197 raft_consensus.cc:729] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ce87b5fa7d7411cb73410ef8a88e84d, State: Initialized, Role: FOLLOWER
I20260928 06:43:04.897820 23197 consensus_queue.cc:260] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:04.898061 23197 ts_tablet_manager.cc:1428] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 06:43:04.901115 23509 tablet_service.cc:1467] Processing CreateTablet for tablet c71e834c54f1468894ace6e9f0f26f06 (DEFAULT_TABLE table=test-table [id=4fc62e1dd53c4749b3110fea8075820b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:04.902218 23509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c71e834c54f1468894ace6e9f0f26f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:04.904335 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.889306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42333 (local address 127.0.0.1:46270)
0928 06:43:04.889622 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:04.889636 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:04.889705 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:04.895799 (+ 6094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:04.895802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:04.895818 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:04.895904 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.895910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.901301 (+ 5391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.901304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.902407 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:04.902418 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.903959 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.903963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:04.904061 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.904086 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 06:43:04.904138 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 06:43:04.904217 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243}
I20260928 06:43:04.905084 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:04.889628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.8.151.194:42333)
0928 06:43:04.895340 (+ 5712us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:04.895351 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:04.895374 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:43:04.895443 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:04.895447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:04.895647 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:04.899874 (+ 4227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:04.900066 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.901150 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.902842 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:04.903828 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:04.904664 (+ 836us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:04.904810 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:04.904861 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 06:43:04.904949 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5607,"thread_start_us":119,"threads_started":1}
I20260928 06:43:04.908402 23377 tablet_service.cc:1467] Processing CreateTablet for tablet c71e834c54f1468894ace6e9f0f26f06 (DEFAULT_TABLE table=test-table [id=4fc62e1dd53c4749b3110fea8075820b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:04.910044 23377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c71e834c54f1468894ace6e9f0f26f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:04.917698 23589 tablet_bootstrap.cc:492] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: Bootstrap starting.
I20260928 06:43:04.919620 23588 tablet_bootstrap.cc:492] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: Bootstrap starting.
I20260928 06:43:04.921447 23589 tablet_bootstrap.cc:654] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:04.922313 23588 tablet_bootstrap.cc:654] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:04.922394 23589 log.cc:824] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:04.923182 23588 log.cc:824] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:04.925132 23589 tablet_bootstrap.cc:492] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: No bootstrap required, opened a new log
I20260928 06:43:04.925329 23589 ts_tablet_manager.cc:1397] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260928 06:43:04.928619 23589 raft_consensus.cc:348] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:04.928925 23589 raft_consensus.cc:374] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:04.928983 23589 raft_consensus.cc:729] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd8fbe08cc1408987b0b2c0efebf0f2, State: Initialized, Role: FOLLOWER
I20260928 06:43:04.929193 23588 tablet_bootstrap.cc:492] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: No bootstrap required, opened a new log
I20260928 06:43:04.929431 23588 ts_tablet_manager.cc:1397] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260928 06:43:04.929508 23589 consensus_queue.cc:260] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:04.930205 23443 heartbeater.cc:503] Master 127.8.151.254:45013 was elected leader, sending a full tablet report...
I20260928 06:43:04.930850 23589 ts_tablet_manager.cc:1428] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260928 06:43:04.933743 23588 raft_consensus.cc:348] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:04.934056 23588 raft_consensus.cc:374] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:04.934110 23588 raft_consensus.cc:729] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a72efc7c91412da07b3f2ee51985b0, State: Initialized, Role: FOLLOWER
I20260928 06:43:04.934618 23588 consensus_queue.cc:260] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:04.935377 23575 heartbeater.cc:503] Master 127.8.151.254:45013 was elected leader, sending a full tablet report...
I20260928 06:43:04.935799 23588 ts_tablet_manager.cc:1428] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260928 06:43:04.945024 23306 tablet.cc:2369] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:43:05.030335 23311 raft_consensus.cc:482] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:43:05.030645 23311 raft_consensus.cc:504] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:05.032075 23311 leader_election.cc:288] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333), f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793)
I20260928 06:43:05.037376 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.032765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:38721 (local address 127.8.151.194:42333)
0928 06:43:05.032894 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:05.032898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:05.033148 (+ 250us) server_negotiation.cc:407] Connection header received
0928 06:43:05.033377 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:05.033380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:05.033430 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:05.033475 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:05.033833 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:05.034552 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:05.036118 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:05.036672 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:05.036760 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:05.036809 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:05.037172 (+ 363us) server_negotiation.cc:299] Negotiation successful
0928 06:43:05.037252 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 06:43:05.037559 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.032806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:40255 (local address 127.8.151.195:33793)
0928 06:43:05.032926 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:05.032931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:05.033284 (+ 353us) server_negotiation.cc:407] Connection header received
0928 06:43:05.033477 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:05.033480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:05.033529 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:05.033586 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:05.033954 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:05.034733 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:05.036266 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:05.036903 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:05.036990 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:05.037038 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:05.037357 (+ 319us) server_negotiation.cc:299] Negotiation successful
0928 06:43:05.037430 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:43:05.037756 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.032337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42333 (local address 127.8.151.193:38721)
0928 06:43:05.033059 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.033073 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:05.033241 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:05.033604 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:05.033607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:05.033623 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:05.033722 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:05.033727 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:05.034688 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:05.034690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:05.035959 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:05.035970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:05.036899 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:05.036902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:05.036993 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:05.037039 (+ 46us) client_negotiation.cc:769] Sending connection context
0928 06:43:05.037553 (+ 514us) client_negotiation.cc:240] Negotiation successful
0928 06:43:05.037632 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":93,"threads_started":1}
I20260928 06:43:05.038323 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.032450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:33793 (local address 127.8.151.193:40255)
0928 06:43:05.033199 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.033213 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:05.033358 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:05.033718 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:05.033721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:05.033736 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:05.033831 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:05.033836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:05.034877 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:05.034880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:05.036120 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:05.036132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:05.037130 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:05.037133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:05.037230 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:05.037251 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 06:43:05.038114 (+ 863us) client_negotiation.cc:240] Negotiation successful
0928 06:43:05.038194 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":57,"threads_started":1}
I20260928 06:43:05.038365 23397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c71e834c54f1468894ace6e9f0f26f06" candidate_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" is_pre_election: true
I20260928 06:43:05.038676 23397 raft_consensus.cc:2454] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ce87b5fa7d7411cb73410ef8a88e84d in term 0.
I20260928 06:43:05.039081 23529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c71e834c54f1468894ace6e9f0f26f06" candidate_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a72efc7c91412da07b3f2ee51985b0" is_pre_election: true
I20260928 06:43:05.039175 23191 leader_election.cc:302] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ce87b5fa7d7411cb73410ef8a88e84d, bdd8fbe08cc1408987b0b2c0efebf0f2; no voters:
I20260928 06:43:05.039408 23529 raft_consensus.cc:2454] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ce87b5fa7d7411cb73410ef8a88e84d in term 0.
I20260928 06:43:05.039484 23311 raft_consensus.cc:2781] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:43:05.039556 23311 raft_consensus.cc:482] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:43:05.039590 23311 raft_consensus.cc:3037] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:05.040776 23311 raft_consensus.cc:504] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:05.041131 23311 leader_election.cc:288] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 1 election: Requested vote from peers bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333), f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793)
I20260928 06:43:05.041407 23397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c71e834c54f1468894ace6e9f0f26f06" candidate_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2"
I20260928 06:43:05.041522 23397 raft_consensus.cc:3037] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:05.041656 23529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c71e834c54f1468894ace6e9f0f26f06" candidate_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a72efc7c91412da07b3f2ee51985b0"
I20260928 06:43:05.041765 23529 raft_consensus.cc:3037] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:05.043200 23529 raft_consensus.cc:2454] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ce87b5fa7d7411cb73410ef8a88e84d in term 1.
I20260928 06:43:05.043551 23193 leader_election.cc:302] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ce87b5fa7d7411cb73410ef8a88e84d, f6a72efc7c91412da07b3f2ee51985b0; no voters:
I20260928 06:43:05.043677 23397 raft_consensus.cc:2454] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ce87b5fa7d7411cb73410ef8a88e84d in term 1.
I20260928 06:43:05.043808 23311 raft_consensus.cc:2781] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:43:05.043964 23311 raft_consensus.cc:686] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 LEADER]: Becoming Leader. State: Replica: 9ce87b5fa7d7411cb73410ef8a88e84d, State: Running, Role: LEADER
I20260928 06:43:05.044234 23311 consensus_queue.cc:237] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:05.047560 23117 catalog_manager.cc:5495] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ce87b5fa7d7411cb73410ef8a88e84d (127.8.151.193). New cstate: current_term: 1 leader_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } health_report { overall_health: HEALTHY } } }
I20260928 06:43:05.062229 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.058186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38914 (local address 127.8.151.193:40065)
0928 06:43:05.058486 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:05.058495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:05.058510 (+ 15us) server_negotiation.cc:407] Connection header received
0928 06:43:05.058594 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:05.058597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:05.058662 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:05.058784 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:43:05.059030 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:05.059808 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:05.060651 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:05.060957 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:05.061067 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:05.061273 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:43:05.061847 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:43:05.061963 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:05.062005 (+ 42us) server_negotiation.cc:299] Negotiation successful
0928 06:43:05.062072 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":117,"threads_started":1}
I20260928 06:43:05.066447 23529 raft_consensus.cc:1261] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0 [term 1 FOLLOWER]: Refusing update from remote peer 9ce87b5fa7d7411cb73410ef8a88e84d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:43:05.066447 23397 raft_consensus.cc:1261] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 1 FOLLOWER]: Refusing update from remote peer 9ce87b5fa7d7411cb73410ef8a88e84d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:43:05.067267 23311 consensus_queue.cc:1035] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:43:05.067461 23596 consensus_queue.cc:1035] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:43:05.077901 23602 mvcc.cc:205] Tried to move back new op lower bound from 7334206607625064448 to 7334206607544119296. Current Snapshot: MvccSnapshot[applied={T|T < 7334206607625064448}]
I20260928 06:43:05.079262 23601 mvcc.cc:205] Tried to move back new op lower bound from 7334206607625064448 to 7334206607544119296. Current Snapshot: MvccSnapshot[applied={T|T < 7334206607625064448}]
I20260928 06:43:05.087867 23113 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:43:05.088076 23113 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:43:05.090816 23113 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Sending DeleteTablet for 3 replicas of tablet c71e834c54f1468894ace6e9f0f26f06
I20260928 06:43:05.091702 23377 tablet_service.cc:1514] Processing DeleteTablet for tablet c71e834c54f1468894ace6e9f0f26f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:05 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:46270
I20260928 06:43:05.092290 23610 tablet_replica.cc:330] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: stopping tablet replica
I20260928 06:43:05.092681 23610 raft_consensus.cc:2229] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:43:05.092990 23509 tablet_service.cc:1514] Processing DeleteTablet for tablet c71e834c54f1468894ace6e9f0f26f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:05 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:49768
I20260928 06:43:05.093081 23610 raft_consensus.cc:2258] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:43:05.093751 23225 tablet_service.cc:1514] Processing DeleteTablet for tablet c71e834c54f1468894ace6e9f0f26f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:05 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:38912
I20260928 06:43:05.094219 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 23155
I20260928 06:43:05.094841 23610 ts_tablet_manager.cc:1905] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:43:05.097662 23610 ts_tablet_manager.cc:1918] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:43:05.097788 23610 log.cc:1196] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/c71e834c54f1468894ace6e9f0f26f06
I20260928 06:43:05.098127 23610 ts_tablet_manager.cc:1939] T c71e834c54f1468894ace6e9f0f26f06 P bdd8fbe08cc1408987b0b2c0efebf0f2: Deleting consensus metadata
I20260928 06:43:05.099409 23611 tablet_replica.cc:330] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: stopping tablet replica
I20260928 06:43:05.099592 23101 catalog_manager.cc:4841] TS bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333): tablet c71e834c54f1468894ace6e9f0f26f06 (table test-table [id=4fc62e1dd53c4749b3110fea8075820b]) successfully deleted
I20260928 06:43:05.099897 23611 raft_consensus.cc:2229] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:43:05.100278 23611 raft_consensus.cc:2258] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 06:43:05.100903 23104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:40065 (error 108)
I20260928 06:43:05.101181 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 23307
I20260928 06:43:05.102955 23611 ts_tablet_manager.cc:1905] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:43:05.103035 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.102743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:40065 (local address 127.0.0.1:38930)
0928 06:43:05.102869 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.102945 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:40065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 06:43:05.103480 23104 catalog_manager.cc:4584] TS 9ce87b5fa7d7411cb73410ef8a88e84d (127.8.151.193:40065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c71e834c54f1468894ace6e9f0f26f06: Network error: Client connection negotiation failed: client connection to 127.8.151.193:40065: connect: Connection refused (error 111)
I20260928 06:43:05.105492 23611 ts_tablet_manager.cc:1918] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:43:05.105585 23611 log.cc:1196] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/c71e834c54f1468894ace6e9f0f26f06
I20260928 06:43:05.105935 23611 ts_tablet_manager.cc:1939] T c71e834c54f1468894ace6e9f0f26f06 P f6a72efc7c91412da07b3f2ee51985b0: Deleting consensus metadata
I20260928 06:43:05.106395 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 23446
I20260928 06:43:05.112434 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 23084
I20260928 06:43:05.118356 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:45013
--webserver_interface=127.8.151.254
--webserver_port=42739
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:45013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:05.229614 23614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:05.229889 23614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:05.229930 23614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:05.234079 23614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:43:05.234144 23614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:05.234162 23614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:05.234179 23614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:43:05.234192 23614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:43:05.239679 23614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:45013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:45013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=42739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:05.240784 23614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:05.241860 23614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:05.250366 23619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:05.250392 23622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:05.250392 23620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:05.250910 23614 server_base.cc:1034] running on GCE node
I20260928 06:43:05.251540 23614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:05.252851 23614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:05.254025 23614 hybrid_clock.cc:648] HybridClock initialized: now 1790577785254006 us; error 35 us; skew 500 ppm
Sep 28 06:43:05 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577785, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:05.257345 23614 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:43:05.258405 23614 webserver.cc:460] Webserver started at http://127.8.151.254:42739/ using document root <none> and password file <none>
I20260928 06:43:05.259033 23614 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:05.259116 23614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:05.263747 23614 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:43:05.266382 23629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:05.267822 23614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:43:05.267946 23614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "90d8dfb9ad58457c94e9c1fbd0132052"
format_stamp: "Formatted at 2026-09-28 06:43:02 on dist-test-slave-gtr8"
server_key: "797658db8a037efbc34b64bd3a09d745"
server_key_iv: "b1ac740d255853e95edd9b8aaa913cfb"
server_key_version: "encryptionkey@0"
I20260928 06:43:05.268597 23614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:05.286350 23614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:05.289645 23614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:05.289820 23614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:05.297991 23614 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:45013
I20260928 06:43:05.298249 23681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:45013 every 8 connection(s)
I20260928 06:43:05.299284 23614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:43:05.303764 23682 sys_catalog.cc:263] Verifying existing consensus state
I20260928 06:43:05.304852 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 23614
I20260928 06:43:05.305348 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:40065
--local_ip_for_outbound_sockets=127.8.151.193
--tserver_master_addrs=127.8.151.254:45013
--webserver_port=40639
--webserver_interface=127.8.151.193
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:43:05.305675 23682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Bootstrap starting.
I20260928 06:43:05.329167 23682 log.cc:824] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:05.343751 23682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:43:05.344211 23682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: Bootstrap complete.
I20260928 06:43:05.349632 23682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:05.350129 23682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d8dfb9ad58457c94e9c1fbd0132052, State: Initialized, Role: FOLLOWER
I20260928 06:43:05.350803 23682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:05.350958 23682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:43:05.351024 23682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:43:05.351122 23682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 2 FOLLOWER]: Advancing to term 3
I20260928 06:43:05.353607 23682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:05.353971 23682 leader_election.cc:302] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90d8dfb9ad58457c94e9c1fbd0132052; no voters:
I20260928 06:43:05.354331 23682 leader_election.cc:288] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 06:43:05.354501 23686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 3 FOLLOWER]: Leader election won for term 3
I20260928 06:43:05.354836 23686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [term 3 LEADER]: Becoming Leader. State: Replica: 90d8dfb9ad58457c94e9c1fbd0132052, State: Running, Role: LEADER
I20260928 06:43:05.355225 23686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } }
I20260928 06:43:05.355649 23682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:43:05.356283 23686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90d8dfb9ad58457c94e9c1fbd0132052. Latest consensus state: current_term: 3 leader_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } } }
I20260928 06:43:05.356487 23686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:05.356639 23687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d8dfb9ad58457c94e9c1fbd0132052" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45013 } } }
I20260928 06:43:05.356724 23687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052 [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:05.360973 23701 catalog_manager.cc:1259] Loaded cluster ID: ef8bede524004b85937964dccb3fb571
I20260928 06:43:05.361038 23701 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: loading cluster ID for follower catalog manager: success
W20260928 06:43:05.362059 23701 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 90d8dfb9ad58457c94e9c1fbd0132052: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260928 06:43:05.362133 23701 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260928 06:43:05.362188 23695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:43:05.364951 23695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:43:05.365473 23695 catalog_manager.cc:1259] Loaded cluster ID: ef8bede524004b85937964dccb3fb571
I20260928 06:43:05.365551 23695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260928 06:43:05.366261 23695 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9d47c679ad google::LogMessage::Fail() at ??:0
@ 0x7f9d47c6a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f9d47c674de google::LogMessage::Flush() at ??:0
@ 0x7f9d47c68a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9d525a6fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9d525a7d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9d525a4579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9d525e8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9d49eb93d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9d49eb9c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f9d49ebb3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9d49eaa512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9d4b4ee6db start_thread at ??:0
@ 0x7f9d4676561f clone at ??:0
W20260928 06:43:05.462796 23684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:05.463040 23684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:05.463100 23684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:05.467025 23684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:05.467079 23684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:05.467180 23684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:43:05.472361 23684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:40065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=40639
--tserver_master_addrs=127.8.151.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:05.473407 23684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:05.474391 23684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:05.483695 23726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:05.483659 23723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:05.484058 23724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:05.484058 23684 server_base.cc:1034] running on GCE node
I20260928 06:43:05.484508 23684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:05.485128 23684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:05.486305 23684 hybrid_clock.cc:648] HybridClock initialized: now 1790577785486282 us; error 33 us; skew 500 ppm
Sep 28 06:43:05 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577785, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:05.489229 23684 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:43:05.490113 23684 webserver.cc:460] Webserver started at http://127.8.151.193:40639/ using document root <none> and password file <none>
I20260928 06:43:05.490640 23684 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:05.490705 23684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:05.497171 23684 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260928 06:43:05.500478 23733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:05.501713 23684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 06:43:05.501819 23684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "9ce87b5fa7d7411cb73410ef8a88e84d"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "0352cc2cb6f82591a4fefdb4401eaf4b"
server_key_iv: "0355c26f738d03cb35c8b95d07296ab4"
server_key_version: "encryptionkey@0"
I20260928 06:43:05.502363 23684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:05.525168 23684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:05.528380 23684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:05.528576 23684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:05.529198 23684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:05.530332 23740 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 06:43:05.534417 23684 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 06:43:05.534473 23684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260928 06:43:05.534543 23684 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 06:43:05.536188 23684 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 06:43:05.536245 23684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260928 06:43:05.536614 23740 tablet_bootstrap.cc:492] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Bootstrap starting.
I20260928 06:43:05.546489 23684 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:40065
I20260928 06:43:05.546547 23847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:40065 every 8 connection(s)
I20260928 06:43:05.548363 23684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:43:05.553447 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 23684
I20260928 06:43:05.553941 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:42333
--local_ip_for_outbound_sockets=127.8.151.194
--tserver_master_addrs=127.8.151.254:45013
--webserver_port=33213
--webserver_interface=127.8.151.194
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:43:05.561259 23740 log.cc:824] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:05.569813 23740 tablet_bootstrap.cc:492] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:43:05.570266 23740 tablet_bootstrap.cc:492] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Bootstrap complete.
I20260928 06:43:05.571595 23740 ts_tablet_manager.cc:1397] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Time spent bootstrapping tablet: real 0.035s user 0.027s sys 0.004s
I20260928 06:43:05.574553 23740 raft_consensus.cc:348] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:05.575174 23740 raft_consensus.cc:729] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ce87b5fa7d7411cb73410ef8a88e84d, State: Initialized, Role: FOLLOWER
I20260928 06:43:05.575740 23740 consensus_queue.cc:260] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:05.576797 23740 ts_tablet_manager.cc:1428] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260928 06:43:05.622360 23850 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:43:05.550631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:34003)
0928 06:43:05.551211 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.551279 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:05.552224 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:05.621337 (+ 69113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":161,"threads_started":1}
W20260928 06:43:05.623024 23848 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:45013 (0 consecutive failures): Network error: Failed to ping master at 127.8.151.254:45013: Client connection negotiation failed: client connection to 127.8.151.254:45013: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 06:43:05.623852 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.623527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:55297)
0928 06:43:05.623695 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.623781 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260928 06:43:05.624665 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.624428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:57565)
0928 06:43:05.624545 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.624601 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260928 06:43:05.624927 23848 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 06:43:05.683823 23851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:05.684100 23851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:05.684160 23851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:05.688257 23851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:05.688326 23851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:05.688406 23851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:43:05.693893 23851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:42333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=33213
--tserver_master_addrs=127.8.151.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:05.695003 23851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:05.696089 23851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:05.705768 23859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:05.705768 23858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:05.705773 23861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:05.706189 23851 server_base.cc:1034] running on GCE node
I20260928 06:43:05.706957 23851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:05.707700 23851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:05.708901 23851 hybrid_clock.cc:648] HybridClock initialized: now 1790577785708858 us; error 63 us; skew 500 ppm
Sep 28 06:43:05 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577785, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:05.711786 23851 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:43:05.712705 23851 webserver.cc:460] Webserver started at http://127.8.151.194:33213/ using document root <none> and password file <none>
I20260928 06:43:05.713248 23851 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:05.713308 23851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:05.717170 23851 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 06:43:05.719218 23868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:05.720526 23851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:43:05.720633 23851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "99bcaaf1c9d9535663646842e196c368"
server_key_iv: "b8c11f7f83fe814c7979327579f24362"
server_key_version: "encryptionkey@0"
I20260928 06:43:05.721210 23851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:05.746803 23851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:05.750263 23851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:05.750439 23851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:05.751109 23851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:05.752060 23851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:05.752110 23851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:05.752154 23851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:05.752183 23851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:05.761977 23851 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:42333
I20260928 06:43:05.762007 23981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:42333 every 8 connection(s)
I20260928 06:43:05.763031 23851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:43:05.767335 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.765222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:35097)
0928 06:43:05.765764 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.766315 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":113,"threads_started":1}
W20260928 06:43:05.767818 23982 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:45013 (0 consecutive failures): Network error: Failed to ping master at 127.8.151.254:45013: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
I20260928 06:43:05.768472 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.768207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:52311)
0928 06:43:05.768334 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.768400 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 06:43:05.769213 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.768996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:39743)
0928 06:43:05.769102 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.769148 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 06:43:05.769377 23982 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 06:43:05.771639 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 23851
I20260928 06:43:05.772125 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:33793
--local_ip_for_outbound_sockets=127.8.151.195
--tserver_master_addrs=127.8.151.254:45013
--webserver_port=43251
--webserver_interface=127.8.151.195
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:05.882345 23985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:05.882586 23985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:05.882634 23985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:05.886631 23985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:05.886696 23985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:05.886755 23985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:43:05.892122 23985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:33793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=43251
--tserver_master_addrs=127.8.151.254:45013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:05.893194 23985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:05.894198 23985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:05.904601 23991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:05.904620 23990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:05.904565 23993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:05.905189 23985 server_base.cc:1034] running on GCE node
I20260928 06:43:05.905627 23985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:05.906309 23985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:05.907464 23985 hybrid_clock.cc:648] HybridClock initialized: now 1790577785907444 us; error 60 us; skew 500 ppm
Sep 28 06:43:05 dist-test-slave-gtr8 krb5kdc[22536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577785, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:05.910238 23985 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:43:05.911177 23985 webserver.cc:460] Webserver started at http://127.8.151.195:43251/ using document root <none> and password file <none>
I20260928 06:43:05.911716 23985 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:05.911782 23985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:05.915632 23985 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260928 06:43:05.917773 24000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:05.919250 23985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:43:05.919368 23985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "f6a72efc7c91412da07b3f2ee51985b0"
format_stamp: "Formatted at 2026-09-28 06:43:03 on dist-test-slave-gtr8"
server_key: "9e615b4fa71e0628ce9e339cb6bd8c0d"
server_key_iv: "6f2750087e36949fccfc53168c76b27c"
server_key_version: "encryptionkey@0"
I20260928 06:43:05.919917 23985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:05.953941 23985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:05.957149 23985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:05.957319 23985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:05.957912 23985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:05.958775 23985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:05.958822 23985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:05.958874 23985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:05.958900 23985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:05.968762 23985 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:33793
I20260928 06:43:05.968878 24113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:33793 every 8 connection(s)
I20260928 06:43:05.969800 23985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:43:05.973888 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.971905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:55691)
0928 06:43:05.972416 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.972948 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":99,"threads_started":1}
W20260928 06:43:05.974349 24114 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:45013 (0 consecutive failures): Network error: Failed to ping master at 127.8.151.254:45013: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
I20260928 06:43:05.974937 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.974691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:38137)
0928 06:43:05.974809 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.974870 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 06:43:05.975575 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:05.975354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:40165)
0928 06:43:05.975457 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:05.975513 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 06:43:05.975781 24114 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 06:43:05.979221 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 23985
I20260928 06:43:06.626169 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:06.625656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:48657)
0928 06:43:06.625945 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:06.626017 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":90,"threads_started":1}
I20260928 06:43:06.770275 24119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:06.769884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:45985)
0928 06:43:06.770115 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:06.770171 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":79,"threads_started":1}
I20260928 06:43:06.976668 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:06.976290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:54727)
0928 06:43:06.976516 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:06.976567 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
I20260928 06:43:07.237576 24121 raft_consensus.cc:482] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:43:07.237803 24121 raft_consensus.cc:504] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:07.239058 24121 leader_election.cc:288] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333), f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793)
Sep 28 06:43:07 dist-test-slave-gtr8 krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577785, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.195@KRBTEST.COM
Sep 28 06:43:07 dist-test-slave-gtr8 krb5kdc[22536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577785, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.194@KRBTEST.COM
I20260928 06:43:07.253667 24123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:07.239333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:33793 (local address 127.8.151.193:41147)
0928 06:43:07.239696 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:07.239712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:07.239834 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:07.241258 (+ 1424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:07.241264 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:07.241725 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:07.242349 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:07.242371 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:07.244205 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:07.244208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:07.245101 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:07.245111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:07.246301 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:07.246304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:07.246374 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:07.246956 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:07.246972 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:07.248697 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:07.251276 (+ 2579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:07.251280 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:07.251282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:07.251582 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:07.251940 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:07.251943 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:07.251946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:07.252082 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:07.252742 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:07.252749 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:07.252963 (+ 214us) client_negotiation.cc:769] Sending connection context
0928 06:43:07.253128 (+ 165us) client_negotiation.cc:240] Negotiation successful
0928 06:43:07.253460 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":7,"spinlock_wait_cycles":10624,"thread_start_us":85,"threads_started":1}
I20260928 06:43:07.253679 24124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:07.239509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:41147 (local address 127.8.151.195:33793)
0928 06:43:07.239821 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:07.239845 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:07.239884 (+ 39us) server_negotiation.cc:407] Connection header received
0928 06:43:07.240549 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:07.240554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:07.240896 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:07.241179 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:07.242538 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:07.244038 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:07.245227 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:07.245953 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:07.246112 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:07.248949 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:07.248957 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:07.248975 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:07.249014 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:07.251116 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:07.251696 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:07.251702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:07.251707 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:07.251835 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:07.252186 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:07.252190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:07.252192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:07.252620 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:07.252742 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:07.253269 (+ 527us) server_negotiation.cc:299] Negotiation successful
0928 06:43:07.253521 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":79,"threads_started":1}
I20260928 06:43:07.254035 24122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:07.239444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42333 (local address 127.8.151.193:45823)
0928 06:43:07.240037 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:07.240050 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:07.240138 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:07.241928 (+ 1790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:07.241931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:07.242098 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:07.242349 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:07.242371 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:07.244344 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:07.244347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:07.245165 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:07.245178 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:07.246093 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:07.246097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:07.246298 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:07.246991 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:07.247004 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:07.249018 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:07.251163 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:07.251170 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:07.251185 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:07.251521 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:07.251908 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:07.251911 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:07.251912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:07.252100 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:07.252908 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:07.252911 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:07.253002 (+ 91us) client_negotiation.cc:769] Sending connection context
0928 06:43:07.253128 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 06:43:07.253918 (+ 790us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":590,"thread_start_us":90,"threads_started":1}
I20260928 06:43:07.255620 24125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:07.239639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:45823 (local address 127.8.151.194:42333)
0928 06:43:07.240377 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:07.240401 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:07.240441 (+ 40us) server_negotiation.cc:407] Connection header received
0928 06:43:07.241280 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:07.241289 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:07.241683 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:07.241979 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:07.243250 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:07.244205 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:07.245279 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:07.245953 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:07.246112 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:07.249243 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:07.249249 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:07.249265 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:07.249301 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:07.251016 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:07.251631 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:07.251637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:07.251644 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:07.251805 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:07.252216 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:07.252219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:07.252221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:07.252741 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:07.252861 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:07.253442 (+ 581us) server_negotiation.cc:299] Negotiation successful
0928 06:43:07.255444 (+ 2002us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":102,"threads_started":1}
W20260928 06:43:07.257264 23736 leader_election.cc:341] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793): Not found: Tablet not found: c71e834c54f1468894ace6e9f0f26f06
W20260928 06:43:07.257822 23734 leader_election.cc:341] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333): Not found: Tablet not found: c71e834c54f1468894ace6e9f0f26f06
I20260928 06:43:07.257939 23734 leader_election.cc:302] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9ce87b5fa7d7411cb73410ef8a88e84d; no voters: bdd8fbe08cc1408987b0b2c0efebf0f2, f6a72efc7c91412da07b3f2ee51985b0
I20260928 06:43:07.258204 24121 raft_consensus.cc:2726] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 06:43:07.627378 24122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:07.626999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:39113)
0928 06:43:07.627211 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:07.627284 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 06:43:07.771484 24126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:07.771015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:44491)
0928 06:43:07.771325 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:07.771386 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":85,"threads_started":1}
I20260928 06:43:07.977813 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:07.977380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:38235)
0928 06:43:07.977657 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:07.977716 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":70,"threads_started":1}
I20260928 06:43:08.628626 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:08.628189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:59411)
0928 06:43:08.628435 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:08.628496 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260928 06:43:08.772610 24129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:08.772193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:49821)
0928 06:43:08.772438 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:08.772498 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
I20260928 06:43:08.806604 24130 raft_consensus.cc:482] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:43:08.806756 24130 raft_consensus.cc:504] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:08.807219 24130 leader_election.cc:288] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333), f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793)
W20260928 06:43:08.808079 23736 leader_election.cc:341] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793): Not found: Tablet not found: c71e834c54f1468894ace6e9f0f26f06
W20260928 06:43:08.808220 23734 leader_election.cc:341] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333): Not found: Tablet not found: c71e834c54f1468894ace6e9f0f26f06
I20260928 06:43:08.808290 23734 leader_election.cc:302] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9ce87b5fa7d7411cb73410ef8a88e84d; no voters: bdd8fbe08cc1408987b0b2c0efebf0f2, f6a72efc7c91412da07b3f2ee51985b0
I20260928 06:43:08.808445 24130 raft_consensus.cc:2726] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 06:43:08.978977 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:08.978533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:51373)
0928 06:43:08.978806 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:08.978868 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":80,"threads_started":1}
I20260928 06:43:09.629948 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:09.629405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:51837)
0928 06:43:09.629776 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:09.629850 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":93,"threads_started":1}
I20260928 06:43:09.773696 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:09.773293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:47401)
0928 06:43:09.773517 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:09.773607 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":81,"threads_started":1}
I20260928 06:43:09.980129 24134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:09.979683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:36161)
0928 06:43:09.979964 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:09.980028 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":99,"threads_started":1}
I20260928 06:43:10.552241 24135 raft_consensus.cc:482] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:43:10.552356 24135 raft_consensus.cc:504] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:10.552757 24135 leader_election.cc:288] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333), f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793)
W20260928 06:43:10.553512 23736 leader_election.cc:341] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793): Not found: Tablet not found: c71e834c54f1468894ace6e9f0f26f06
W20260928 06:43:10.553647 23734 leader_election.cc:341] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333): Not found: Tablet not found: c71e834c54f1468894ace6e9f0f26f06
I20260928 06:43:10.553754 23734 leader_election.cc:302] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9ce87b5fa7d7411cb73410ef8a88e84d; no voters: bdd8fbe08cc1408987b0b2c0efebf0f2, f6a72efc7c91412da07b3f2ee51985b0
I20260928 06:43:10.553889 24135 raft_consensus.cc:2726] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 06:43:10.631394 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:10.630864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:58667)
0928 06:43:10.631237 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:10.631297 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":94,"threads_started":1}
I20260928 06:43:10.775009 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:10.774492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:58683)
0928 06:43:10.774849 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:10.774912 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":106,"threads_started":1}
I20260928 06:43:10.981353 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:10.980899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:51305)
0928 06:43:10.981198 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:10.981259 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":94,"threads_started":1}
I20260928 06:43:11.632464 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:11.632065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:43749)
0928 06:43:11.632315 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:11.632370 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260928 06:43:11.776234 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:11.775794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:60569)
0928 06:43:11.776047 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:11.776118 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":62,"threads_started":1}
I20260928 06:43:11.982448 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:11.982041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:33065)
0928 06:43:11.982297 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:11.982353 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
I20260928 06:43:12.633693 24142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:12.633268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:39421)
0928 06:43:12.633533 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:12.633597 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260928 06:43:12.777642 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:12.777123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:45479)
0928 06:43:12.777478 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:12.777544 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":82,"threads_started":1}
I20260928 06:43:12.983737 24144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:12.983250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:44065)
0928 06:43:12.983573 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:12.983639 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":84,"threads_started":1}
I20260928 06:43:13.635082 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:13.634645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:54601)
0928 06:43:13.634927 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:13.634988 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":78,"threads_started":1}
I20260928 06:43:13.778940 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:13.778485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:34081)
0928 06:43:13.778779 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:13.778843 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":71,"threads_started":1}
I20260928 06:43:13.985002 24147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:13.984579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:33909)
0928 06:43:13.984837 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:13.984898 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260928 06:43:14.636268 24148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:14.635865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:60297)
0928 06:43:14.636119 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:14.636174 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260928 06:43:14.780138 24149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:14.779737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:59451)
0928 06:43:14.779987 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:14.780041 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
I20260928 06:43:14.817061 24150 raft_consensus.cc:482] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:43:14.817179 24150 raft_consensus.cc:504] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd8fbe08cc1408987b0b2c0efebf0f2" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42333 } } peers { permanent_uuid: "f6a72efc7c91412da07b3f2ee51985b0" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 33793 } } peers { permanent_uuid: "9ce87b5fa7d7411cb73410ef8a88e84d" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 40065 } }
I20260928 06:43:14.817597 24150 leader_election.cc:288] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333), f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793)
W20260928 06:43:14.818331 23736 leader_election.cc:341] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6a72efc7c91412da07b3f2ee51985b0 (127.8.151.195:33793): Not found: Tablet not found: c71e834c54f1468894ace6e9f0f26f06
W20260928 06:43:14.818457 23734 leader_election.cc:341] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bdd8fbe08cc1408987b0b2c0efebf0f2 (127.8.151.194:42333): Not found: Tablet not found: c71e834c54f1468894ace6e9f0f26f06
I20260928 06:43:14.818543 23734 leader_election.cc:302] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9ce87b5fa7d7411cb73410ef8a88e84d; no voters: bdd8fbe08cc1408987b0b2c0efebf0f2, f6a72efc7c91412da07b3f2ee51985b0
I20260928 06:43:14.818671 24150 raft_consensus.cc:2726] T c71e834c54f1468894ace6e9f0f26f06 P 9ce87b5fa7d7411cb73410ef8a88e84d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 06:43:14.986079 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:14.985674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:35445)
0928 06:43:14.985930 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:14.985984 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260928 06:43:15.637553 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:15.637135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:47057)
0928 06:43:15.637401 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:15.637456 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":81,"threads_started":1}
I20260928 06:43:15.781364 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:15.780925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:36179)
0928 06:43:15.781207 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:15.781267 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":84,"threads_started":1}
I20260928 06:43:15.987375 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:15.986884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:38555)
0928 06:43:15.987201 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:15.987263 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":105,"threads_started":1}
I20260928 06:43:16.638736 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:16.638316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:34535)
0928 06:43:16.638573 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:16.638628 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":80,"threads_started":1}
I20260928 06:43:16.782563 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:16.782137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:39377)
0928 06:43:16.782407 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:16.782467 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":80,"threads_started":1}
I20260928 06:43:16.988478 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:16.988063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:38737)
0928 06:43:16.988332 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:16.988386 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
I20260928 06:43:17.639861 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:17.639470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:40211)
0928 06:43:17.639709 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:17.639760 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":73,"threads_started":1}
I20260928 06:43:17.783732 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:17.783329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:53197)
0928 06:43:17.783578 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:17.783635 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
I20260928 06:43:17.989650 24160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:17.989236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:59277)
0928 06:43:17.989497 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:17.989555 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260928 06:43:18.641194 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:18.640690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:42011)
0928 06:43:18.641047 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:18.641100 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":97,"threads_started":1}
I20260928 06:43:18.785051 24162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:18.784571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:37941)
0928 06:43:18.784850 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:18.784927 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":75,"threads_started":1}
I20260928 06:43:18.990757 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:18.990367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:60499)
0928 06:43:18.990610 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:18.990662 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260928 06:43:19.642367 24164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:19.641938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:45491)
0928 06:43:19.642192 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:19.642249 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
I20260928 06:43:19.786252 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:19.785806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:40553)
0928 06:43:19.786065 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:19.786130 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260928 06:43:19.992080 24166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:19.991645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:41749)
0928 06:43:19.991921 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:19.991982 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260928 06:43:20.643831 24167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:20.643370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.193:58799)
0928 06:43:20.643671 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:20.643732 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":90,"threads_started":1}
I20260928 06:43:20.787662 24168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:20.787248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.194:42045)
0928 06:43:20.787502 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:20.787564 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260928 06:43:20.988629 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 23684
I20260928 06:43:20.993549 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:20.993066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45013 (local address 127.8.151.195:32979)
0928 06:43:20.993361 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:20.993433 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":75,"threads_started":1}
I20260928 06:43:20.995174 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 23851
I20260928 06:43:21.001677 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 23985
2026-09-28T06:43:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19527 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:43:21 dist-test-slave-gtr8 krb5kdc[24172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:43:21 dist-test-slave-gtr8 krb5kdc[24172](info): set up 2 sockets
Sep 28 06:43:21 dist-test-slave-gtr8 krb5kdc[24172](info): commencing operation
krb5kdc: starting...
W20260928 06:43:21.984932 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:43:22 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:43:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:43:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:43:22.122093 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39155
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:39155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:22.234174 24185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:22.234447 24185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:22.234500 24185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:22.238590 24185 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 06:43:22.238646 24185 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 06:43:22.238667 24185 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 06:43:22.238683 24185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:43:22.238708 24185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:22.238741 24185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:22.238768 24185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:43:22.238796 24185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:43:22.245018 24185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:39155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:22.246359 24185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:22.247445 24185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:22.255970 24193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:22.255995 24190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:22.256083 24191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:22.256472 24185 server_base.cc:1034] running on GCE node
I20260928 06:43:22.257009 24185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:22.258100 24185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:22.259244 24185 hybrid_clock.cc:648] HybridClock initialized: now 1790577802259221 us; error 41 us; skew 500 ppm
Sep 28 06:43:22 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577802, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:22.262070 24185 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:43:22.263008 24185 webserver.cc:460] Webserver started at http://127.8.151.254:41135/ using document root <none> and password file <none>
I20260928 06:43:22.263578 24185 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:22.263648 24185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:22.263846 24185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:43:22.265851 24185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "3f51f5b0dc6d435f8dee647b2a20372a"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "72dd8593cf991088a6206dc56286dda4"
server_key_iv: "245dd6f5fff0e15153ff14e10bfcbddb"
server_key_version: "encryptionkey@0"
I20260928 06:43:22.266393 24185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "3f51f5b0dc6d435f8dee647b2a20372a"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "72dd8593cf991088a6206dc56286dda4"
server_key_iv: "245dd6f5fff0e15153ff14e10bfcbddb"
server_key_version: "encryptionkey@0"
I20260928 06:43:22.270109 24185 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260928 06:43:22.272578 24200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:22.273808 24185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:43:22.273936 24185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "3f51f5b0dc6d435f8dee647b2a20372a"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "72dd8593cf991088a6206dc56286dda4"
server_key_iv: "245dd6f5fff0e15153ff14e10bfcbddb"
server_key_version: "encryptionkey@0"
I20260928 06:43:22.274029 24185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:22.299685 24185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:22.302752 24185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:22.302933 24185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:22.310205 24185 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:39155
I20260928 06:43:22.310464 24252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:39155 every 8 connection(s)
I20260928 06:43:22.311501 24185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:43:22.314306 24253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:22.318815 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 24185
I20260928 06:43:22.318943 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:43:22.319274 8799 external_mini_cluster.cc:1428] Setting key 58f7afb9e5b33aa28c0a47ef48acf78e
I20260928 06:43:22.320045 24253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Bootstrap starting.
I20260928 06:43:22.322520 24253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:22.323485 24253 log.cc:824] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:22.325846 24253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: No bootstrap required, opened a new log
Sep 28 06:43:22 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:22.329947 24253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:43:22.330233 24253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:22.330313 24253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f51f5b0dc6d435f8dee647b2a20372a, State: Initialized, Role: FOLLOWER
I20260928 06:43:22.330817 24253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:43:22.331032 24253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:43:22.331110 24253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:43:22.331269 24253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:22.332265 24253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:43:22.332633 24253 leader_election.cc:302] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f51f5b0dc6d435f8dee647b2a20372a; no voters:
I20260928 06:43:22.333084 24253 leader_election.cc:288] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:43:22.333199 24258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:43:22.333433 24258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 1 LEADER]: Becoming Leader. State: Replica: 3f51f5b0dc6d435f8dee647b2a20372a, State: Running, Role: LEADER
I20260928 06:43:22.333772 24258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:43:22.334283 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:22.320715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36910 (local address 127.8.151.254:39155)
0928 06:43:22.321165 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:22.321182 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:22.321224 (+ 42us) server_negotiation.cc:407] Connection header received
0928 06:43:22.322028 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:22.322052 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:22.322458 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:22.322769 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:22.323601 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:22.324763 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:22.325607 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:22.326132 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:22.326284 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:22.328655 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:22.328660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:22.328674 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:22.328702 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:22.330776 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:22.331337 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:22.331344 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:22.331351 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:22.331531 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:22.331839 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:22.331842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:22.331844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:22.332403 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:22.332604 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:22.332915 (+ 311us) server_negotiation.cc:299] Negotiation successful
0928 06:43:22.333240 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":120,"threads_started":1}
I20260928 06:43:22.334753 24253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:43:22.336364 24260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } } }
I20260928 06:43:22.336485 24260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:22.336795 24259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f51f5b0dc6d435f8dee647b2a20372a. Latest consensus state: current_term: 1 leader_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } } }
I20260928 06:43:22.336882 24259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:22.336989 24272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:43:22.340094 24272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:43:22.345378 24272 catalog_manager.cc:1347] Generated new cluster ID: 8b5a88b2ef2b447985f35f88716c6942
I20260928 06:43:22.345448 24272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:43:22.359575 24272 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:43:22.360505 24272 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:43:22.364964 24272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Generated new TSK 0
I20260928 06:43:22.365650 24272 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:43:22.401525 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39155
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:22.545301 24279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:22.545569 24279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:22.545615 24279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:22.549791 24279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:22.549855 24279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:22.549938 24279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:43:22.555858 24279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:22.557350 24279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:22.558704 24279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:22.570031 24287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:22.570278 24279 server_base.cc:1034] running on GCE node
W20260928 06:43:22.570031 24285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:22.570048 24284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:22.570820 24279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:22.571516 24279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:22.572680 24279 hybrid_clock.cc:648] HybridClock initialized: now 1790577802572659 us; error 32 us; skew 500 ppm
Sep 28 06:43:22 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577802, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:22.575683 24279 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:43:22.576704 24279 webserver.cc:460] Webserver started at http://127.8.151.193:39103/ using document root <none> and password file <none>
I20260928 06:43:22.577328 24279 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:22.577405 24279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:22.577618 24279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:43:22.580672 24279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "18ca0400c67b46e3aa45e0e4f355cc0f"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "e1fa75c648384d39a20a390f09a7fda1"
server_key_iv: "decd9a2064effe14422f08b3ea286632"
server_key_version: "encryptionkey@0"
I20260928 06:43:22.581698 24279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "18ca0400c67b46e3aa45e0e4f355cc0f"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "e1fa75c648384d39a20a390f09a7fda1"
server_key_iv: "decd9a2064effe14422f08b3ea286632"
server_key_version: "encryptionkey@0"
I20260928 06:43:22.585925 24279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260928 06:43:22.589094 24294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:22.590399 24279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:43:22.590498 24279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "18ca0400c67b46e3aa45e0e4f355cc0f"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "e1fa75c648384d39a20a390f09a7fda1"
server_key_iv: "decd9a2064effe14422f08b3ea286632"
server_key_version: "encryptionkey@0"
I20260928 06:43:22.590605 24279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:22.614302 24279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:22.617899 24279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:22.618093 24279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:22.618788 24279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:22.619891 24279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:22.619951 24279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:22.620011 24279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:22.620040 24279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:22.630465 24279 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:33611
I20260928 06:43:22.630681 24407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:33611 every 8 connection(s)
I20260928 06:43:22.631791 24279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:43:22.638645 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 24279
I20260928 06:43:22.638762 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:43:22.639057 8799 external_mini_cluster.cc:1428] Setting key cbd05fec6212671388201325238dd78b
Sep 28 06:43:22 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577802, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:22.646412 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:22.633676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:56319 (local address 127.8.151.254:39155)
0928 06:43:22.633822 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:22.633826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:22.634983 (+ 1157us) server_negotiation.cc:407] Connection header received
0928 06:43:22.636025 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:22.636030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:22.636089 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:22.636186 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:22.637365 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:22.638003 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:22.639349 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:22.639687 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:22.639790 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:22.642523 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:22.642526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:22.642529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:22.642554 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:22.644076 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:22.644623 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:22.644628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:22.644630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:22.644726 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:22.645097 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:22.645100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:22.645102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:22.645395 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:22.645497 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:22.646046 (+ 549us) server_negotiation.cc:299] Negotiation successful
0928 06:43:22.646202 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 06:43:22.647251 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:22.634092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:56319)
0928 06:43:22.634834 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:22.634878 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:22.635809 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:22.636325 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:22.636332 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:22.636686 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:22.637227 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:22.637241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:22.638122 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:22.638125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:22.639212 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:22.639224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:22.639863 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:22.639867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:22.640061 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:22.640684 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:22.640702 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:22.642362 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:22.644206 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:22.644212 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:22.644228 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:22.644508 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:22.644852 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:22.644855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:22.644857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:22.645014 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:22.645504 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:22.645510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:22.645788 (+ 278us) client_negotiation.cc:769] Sending connection context
0928 06:43:22.645967 (+ 179us) client_negotiation.cc:240] Negotiation successful
0928 06:43:22.646280 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":156,"threads_started":1}
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:43:22.648693 24408 heartbeater.cc:347] Connected to a master server at 127.8.151.254:39155
I20260928 06:43:22.648977 24408 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:22.649504 24408 heartbeater.cc:511] Master 127.8.151.254:39155 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:43:22.651190 24217 ts_manager.cc:194] Registered new tserver with Master: 18ca0400c67b46e3aa45e0e4f355cc0f (127.8.151.193:33611)
I20260928 06:43:22.652796 24217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:56319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:43:22.668118 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39155
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:22.787163 24413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:22.787551 24413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:22.787616 24413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:22.793685 24413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:22.793761 24413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:22.793851 24413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:43:22.801667 24413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:22.802739 24413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:22.803818 24413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:22.814086 24421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:22.814129 24418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:22.814402 24419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:22.814947 24413 server_base.cc:1034] running on GCE node
I20260928 06:43:22.815596 24413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:22.816213 24413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:22.817384 24413 hybrid_clock.cc:648] HybridClock initialized: now 1790577802817337 us; error 61 us; skew 500 ppm
Sep 28 06:43:22 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577802, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:22.820068 24413 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:43:22.820894 24413 webserver.cc:460] Webserver started at http://127.8.151.194:39433/ using document root <none> and password file <none>
I20260928 06:43:22.821421 24413 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:22.821485 24413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:22.821678 24413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:43:22.823838 24413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "54aab4f020e6407e8a143ce6eb56d9bc"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "549089452f305a42073c7ebfd3c45fd7"
server_key_iv: "82e94efe59702be7324d588581a4c883"
server_key_version: "encryptionkey@0"
I20260928 06:43:22.824432 24413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "54aab4f020e6407e8a143ce6eb56d9bc"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "549089452f305a42073c7ebfd3c45fd7"
server_key_iv: "82e94efe59702be7324d588581a4c883"
server_key_version: "encryptionkey@0"
I20260928 06:43:22.828431 24413 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260928 06:43:22.830961 24428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:22.832274 24413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:43:22.832391 24413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "54aab4f020e6407e8a143ce6eb56d9bc"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "549089452f305a42073c7ebfd3c45fd7"
server_key_iv: "82e94efe59702be7324d588581a4c883"
server_key_version: "encryptionkey@0"
I20260928 06:43:22.832508 24413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:22.859709 24413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:22.862864 24413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:22.863057 24413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:22.863706 24413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:22.864562 24413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:22.864617 24413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:22.864669 24413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:22.864697 24413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:22.874001 24413 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:45037
I20260928 06:43:22.874053 24541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:45037 every 8 connection(s)
I20260928 06:43:22.874938 24413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:43:22.885099 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 24413
I20260928 06:43:22.885224 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:43:22.885516 8799 external_mini_cluster.cc:1428] Setting key 7ebaa36f051a70682d165495f9ee75fd
Sep 28 06:43:22 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577802, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:22.890003 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:22.876941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:34481 (local address 127.8.151.254:39155)
0928 06:43:22.878331 (+ 1390us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:22.878335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:22.878401 (+ 66us) server_negotiation.cc:407] Connection header received
0928 06:43:22.879650 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:22.879654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:22.879711 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:22.879827 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:22.881305 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:22.882097 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:22.883411 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:22.883639 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:22.883724 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:22.886703 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:22.886706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:22.886709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:22.886733 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:22.888057 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:22.888527 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:22.888530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:22.888532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:22.888604 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:22.888918 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:22.888921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:22.888923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:22.889095 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:22.889237 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:22.889724 (+ 487us) server_negotiation.cc:299] Negotiation successful
0928 06:43:22.889847 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1296}
I20260928 06:43:22.890895 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:22.877375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:34481)
0928 06:43:22.878234 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:22.878297 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:22.879379 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:22.880040 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:22.880050 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:22.880536 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:22.881165 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:22.881182 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:22.882221 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:22.882224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:22.883266 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:22.883279 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:22.883789 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:22.883792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:22.883996 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:22.884658 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:22.884676 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:22.886537 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:22.888167 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:22.888173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:22.888188 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:22.888437 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:22.888684 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:22.888687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:22.888690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:22.888834 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:22.889233 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:22.889239 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:22.889501 (+ 262us) client_negotiation.cc:769] Sending connection context
0928 06:43:22.889666 (+ 165us) client_negotiation.cc:240] Negotiation successful
0928 06:43:22.889991 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":116,"threads_started":1}
I20260928 06:43:22.892694 24542 heartbeater.cc:347] Connected to a master server at 127.8.151.254:39155
I20260928 06:43:22.892961 24542 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:22.893553 24542 heartbeater.cc:511] Master 127.8.151.254:39155 requested a full tablet report, sending...
I20260928 06:43:22.894696 24217 ts_manager.cc:194] Registered new tserver with Master: 54aab4f020e6407e8a143ce6eb56d9bc (127.8.151.194:45037)
I20260928 06:43:22.895290 24217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:34481
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:43:22.912264 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39155
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:23.032101 24547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:23.032351 24547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:23.032420 24547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:23.036531 24547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:23.036586 24547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:23.036662 24547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:43:23.042085 24547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:23.043300 24547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:23.044332 24547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:23.053715 24555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:23.053668 24552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:23.053668 24553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:23.053997 24547 server_base.cc:1034] running on GCE node
I20260928 06:43:23.054502 24547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:23.055119 24547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:23.056344 24547 hybrid_clock.cc:648] HybridClock initialized: now 1790577803056318 us; error 42 us; skew 500 ppm
Sep 28 06:43:23 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577803, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:23.059090 24547 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:43:23.060001 24547 webserver.cc:460] Webserver started at http://127.8.151.195:39525/ using document root <none> and password file <none>
I20260928 06:43:23.060549 24547 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:23.060616 24547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:23.060829 24547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:43:23.062857 24547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "e799d9ba35154838b05533257e3de6cc"
format_stamp: "Formatted at 2026-09-28 06:43:23 on dist-test-slave-gtr8"
server_key: "4d6869c7f7ff7842de004974f45e9556"
server_key_iv: "96a437a92370187a2ef4fe4e6aaf8a60"
server_key_version: "encryptionkey@0"
I20260928 06:43:23.063436 24547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "e799d9ba35154838b05533257e3de6cc"
format_stamp: "Formatted at 2026-09-28 06:43:23 on dist-test-slave-gtr8"
server_key: "4d6869c7f7ff7842de004974f45e9556"
server_key_iv: "96a437a92370187a2ef4fe4e6aaf8a60"
server_key_version: "encryptionkey@0"
I20260928 06:43:23.067291 24547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260928 06:43:23.069833 24562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:23.071166 24547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 06:43:23.071281 24547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "e799d9ba35154838b05533257e3de6cc"
format_stamp: "Formatted at 2026-09-28 06:43:23 on dist-test-slave-gtr8"
server_key: "4d6869c7f7ff7842de004974f45e9556"
server_key_iv: "96a437a92370187a2ef4fe4e6aaf8a60"
server_key_version: "encryptionkey@0"
I20260928 06:43:23.071390 24547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:23.081328 24547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:23.084769 24547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:23.084949 24547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:23.085585 24547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:23.086515 24547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:23.086575 24547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:23.086627 24547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:23.086653 24547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:23.096415 24547 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:44671
I20260928 06:43:23.096444 24675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:44671 every 8 connection(s)
I20260928 06:43:23.097464 24547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:43:23.098599 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 24547
I20260928 06:43:23.098717 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:43:23.099006 8799 external_mini_cluster.cc:1428] Setting key 674243edddd55268f42a635ede74bf7c
Sep 28 06:43:23 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577803, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:23.111275 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.099257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:35403 (local address 127.8.151.254:39155)
0928 06:43:23.099400 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:23.099405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:23.101249 (+ 1844us) server_negotiation.cc:407] Connection header received
0928 06:43:23.102112 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:23.102116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:23.102168 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:23.102249 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:23.103389 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.104215 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.105279 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.105624 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.105721 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.107952 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:23.107956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:23.107959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:23.107985 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:23.109416 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:23.109857 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:23.109860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:23.109862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:23.109950 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:23.110220 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:23.110223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:23.110225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:23.110444 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:23.110535 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:23.110954 (+ 419us) server_negotiation.cc:299] Negotiation successful
0928 06:43:23.111070 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 06:43:23.112208 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.099683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:35403)
0928 06:43:23.101132 (+ 1449us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:23.101173 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:23.101930 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:23.102386 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:23.102393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:23.102750 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:23.103262 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.103277 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.104314 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.104316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.105172 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.105181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.105744 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.105746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.105881 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.106402 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:23.106426 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:23.107799 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:23.109518 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:23.109522 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:23.109533 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:23.109762 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:23.110023 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:23.110025 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:23.110026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:23.110143 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:23.110522 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:23.110527 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:23.110740 (+ 213us) client_negotiation.cc:769] Sending connection context
0928 06:43:23.110878 (+ 138us) client_negotiation.cc:240] Negotiation successful
0928 06:43:23.111123 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1262,"thread_start_us":138,"threads_started":1}
I20260928 06:43:23.113353 24676 heartbeater.cc:347] Connected to a master server at 127.8.151.254:39155
I20260928 06:43:23.113596 24676 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:23.114126 24676 heartbeater.cc:511] Master 127.8.151.254:39155 requested a full tablet report, sending...
I20260928 06:43:23.115121 24217 ts_manager.cc:194] Registered new tserver with Master: e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671)
I20260928 06:43:23.115717 24217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:35403
I20260928 06:43:23.124852 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:43:23.135195 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.126972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36926 (local address 127.8.151.254:39155)
0928 06:43:23.127083 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:23.127087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:23.127388 (+ 301us) server_negotiation.cc:407] Connection header received
0928 06:43:23.127467 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:23.127470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:23.127525 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:23.127610 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:23.128237 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.129001 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.129911 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.130230 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.130343 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.131776 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:23.131780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:23.131783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:23.131812 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:23.133433 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:23.133824 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:23.133828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:23.133830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:23.133927 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:23.134141 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:23.134144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:23.134146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:23.134472 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:23.134618 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:23.134890 (+ 272us) server_negotiation.cc:299] Negotiation successful
0928 06:43:23.135009 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 06:43:23.138242 24217 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:43:23.141435 24217 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:43:23.157610 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.150395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:33611 (local address 127.0.0.1:43368)
0928 06:43:23.150849 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:23.150897 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:23.151021 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:23.152530 (+ 1509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:23.152536 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:23.152557 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:23.152711 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.152719 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.154255 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.154258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.155948 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.155960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.157062 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.157066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.157176 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.157216 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 06:43:23.157304 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 06:43:23.157433 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":147,"thread_start_us":71,"threads_started":1}
I20260928 06:43:23.157917 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.150907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40838 (local address 127.8.151.194:45037)
0928 06:43:23.151273 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:23.151284 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:23.152053 (+ 769us) server_negotiation.cc:407] Connection header received
0928 06:43:23.152148 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:23.152153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:23.152358 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:23.152520 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:23.152908 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.154116 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.156091 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.156922 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.157030 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.157194 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:23.157677 (+ 483us) server_negotiation.cc:299] Negotiation successful
0928 06:43:23.157787 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":141,"threads_started":1}
I20260928 06:43:23.157610 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.151470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:45037 (local address 127.0.0.1:40838)
0928 06:43:23.151939 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:23.151951 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:23.152038 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:23.152698 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:23.152701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:23.152715 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:23.152792 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.152798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.154255 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.154258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.155948 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.155960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.157254 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.157257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.157343 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.157361 (+ 18us) client_negotiation.cc:769] Sending connection context
0928 06:43:23.157430 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 06:43:23.157484 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":117,"thread_start_us":88,"threads_started":1}
I20260928 06:43:23.158766 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.151103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:44671 (local address 127.0.0.1:46726)
0928 06:43:23.151366 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:23.151378 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:23.151468 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:23.152884 (+ 1416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:23.152887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:23.152905 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:23.152993 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.152999 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.155280 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.155284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.156507 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.156518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.158372 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.158375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.158489 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.158514 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 06:43:23.158577 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 06:43:23.158645 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":126,"threads_started":1}
I20260928 06:43:23.159017 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.151711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.8.151.193:33611)
0928 06:43:23.152073 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:23.152080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:23.152099 (+ 19us) server_negotiation.cc:407] Connection header received
0928 06:43:23.152159 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:23.152163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:23.152358 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:23.152871 (+ 513us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:23.153016 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.154127 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.156091 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.156932 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.158596 (+ 1664us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.158738 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:23.158786 (+ 48us) server_negotiation.cc:299] Negotiation successful
0928 06:43:23.158876 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":82,"threads_started":1}
I20260928 06:43:23.159181 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.151666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.8.151.195:44671)
0928 06:43:23.152285 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:23.152296 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:23.152318 (+ 22us) server_negotiation.cc:407] Connection header received
0928 06:43:23.152402 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:23.152407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:23.152589 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:23.152726 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:23.153117 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.155074 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.156644 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.158243 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.158349 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.158497 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:23.158918 (+ 421us) server_negotiation.cc:299] Negotiation successful
0928 06:43:23.159022 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":106,"threads_started":1}
I20260928 06:43:23.161556 24342 tablet_service.cc:1467] Processing CreateTablet for tablet 0551dd925e5043918cde3acf1addf223 (DEFAULT_TABLE table=test-table [id=810a51e49c3947b3828e5bcd61445e7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:23.161556 24476 tablet_service.cc:1467] Processing CreateTablet for tablet 0551dd925e5043918cde3acf1addf223 (DEFAULT_TABLE table=test-table [id=810a51e49c3947b3828e5bcd61445e7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:23.162704 24342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0551dd925e5043918cde3acf1addf223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:23.162704 24476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0551dd925e5043918cde3acf1addf223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:23.162557 24610 tablet_service.cc:1467] Processing CreateTablet for tablet 0551dd925e5043918cde3acf1addf223 (DEFAULT_TABLE table=test-table [id=810a51e49c3947b3828e5bcd61445e7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:23.163594 24610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0551dd925e5043918cde3acf1addf223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:23.170089 24693 tablet_bootstrap.cc:492] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Bootstrap starting.
I20260928 06:43:23.171649 24694 tablet_bootstrap.cc:492] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: Bootstrap starting.
I20260928 06:43:23.172513 24693 tablet_bootstrap.cc:654] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:23.173375 24693 log.cc:824] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:23.174105 24695 tablet_bootstrap.cc:492] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: Bootstrap starting.
I20260928 06:43:23.174158 24694 tablet_bootstrap.cc:654] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:23.174983 24694 log.cc:824] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:23.175392 24693 tablet_bootstrap.cc:492] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: No bootstrap required, opened a new log
I20260928 06:43:23.175623 24693 ts_tablet_manager.cc:1397] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260928 06:43:23.176796 24695 tablet_bootstrap.cc:654] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:23.177009 24694 tablet_bootstrap.cc:492] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: No bootstrap required, opened a new log
I20260928 06:43:23.177204 24694 ts_tablet_manager.cc:1397] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 06:43:23.177740 24695 log.cc:824] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:23.179791 24695 tablet_bootstrap.cc:492] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: No bootstrap required, opened a new log
I20260928 06:43:23.179991 24695 ts_tablet_manager.cc:1397] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:43:23.180105 24693 raft_consensus.cc:348] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:23.180384 24693 raft_consensus.cc:374] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:23.180440 24693 raft_consensus.cc:729] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18ca0400c67b46e3aa45e0e4f355cc0f, State: Initialized, Role: FOLLOWER
I20260928 06:43:23.180922 24693 consensus_queue.cc:260] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:23.181043 24694 raft_consensus.cc:348] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:23.181290 24694 raft_consensus.cc:374] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:23.181332 24694 raft_consensus.cc:729] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e799d9ba35154838b05533257e3de6cc, State: Initialized, Role: FOLLOWER
I20260928 06:43:23.181727 24694 consensus_queue.cc:260] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:23.181903 24693 ts_tablet_manager.cc:1428] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260928 06:43:23.182240 24408 heartbeater.cc:503] Master 127.8.151.254:39155 was elected leader, sending a full tablet report...
I20260928 06:43:23.182646 24694 ts_tablet_manager.cc:1428] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260928 06:43:23.182964 24676 heartbeater.cc:503] Master 127.8.151.254:39155 was elected leader, sending a full tablet report...
I20260928 06:43:23.183755 24695 raft_consensus.cc:348] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:23.184036 24695 raft_consensus.cc:374] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:23.184098 24695 raft_consensus.cc:729] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54aab4f020e6407e8a143ce6eb56d9bc, State: Initialized, Role: FOLLOWER
I20260928 06:43:23.184499 24695 consensus_queue.cc:260] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:23.185397 24695 ts_tablet_manager.cc:1428] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:43:23.185763 24542 heartbeater.cc:503] Master 127.8.151.254:39155 was elected leader, sending a full tablet report...
I20260928 06:43:23.329043 24701 raft_consensus.cc:482] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:43:23.329249 24701 raft_consensus.cc:504] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:23.330399 24701 leader_election.cc:288] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18ca0400c67b46e3aa45e0e4f355cc0f (127.8.151.193:33611), e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671)
I20260928 06:43:23.335233 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.330912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:44671 (local address 127.8.151.194:49641)
0928 06:43:23.331222 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:23.331234 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:23.331317 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:23.331538 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:23.331541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:23.331556 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:23.331663 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.331668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.332373 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.332376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.333804 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.333815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.334693 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.334697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.334848 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.334877 (+ 29us) client_negotiation.cc:769] Sending connection context
0928 06:43:23.334974 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 06:43:23.335048 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":113,"thread_start_us":74,"threads_started":1}
I20260928 06:43:23.335234 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.331214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:49641 (local address 127.8.151.195:44671)
0928 06:43:23.331355 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:23.331360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:23.331373 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:43:23.331421 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:23.331423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:23.331458 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:23.331517 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:23.331760 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.332262 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.333950 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.334573 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.334695 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.334751 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:23.335004 (+ 253us) server_negotiation.cc:299] Negotiation successful
0928 06:43:23.335083 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 06:43:23.335671 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.330635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:33611 (local address 127.8.151.194:50735)
0928 06:43:23.330790 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:23.330808 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:23.330928 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:23.331200 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:23.331203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:23.331223 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:23.331345 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.331352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.332354 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.332358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.333804 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:23.333815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.335287 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.335292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:23.335416 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.335437 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 06:43:23.335506 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 06:43:23.335564 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":88}
I20260928 06:43:23.335819 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.330793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:50735 (local address 127.8.151.193:33611)
0928 06:43:23.330933 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:23.330937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:23.330956 (+ 19us) server_negotiation.cc:407] Connection header received
0928 06:43:23.331017 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:23.331020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:23.331071 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:23.331163 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:23.331484 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.332232 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.334587 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.335121 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.335262 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.335325 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:23.335634 (+ 309us) server_negotiation.cc:299] Negotiation successful
0928 06:43:23.335710 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 06:43:23.336272 24630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0551dd925e5043918cde3acf1addf223" candidate_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e799d9ba35154838b05533257e3de6cc" is_pre_election: true
I20260928 06:43:23.336318 24362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0551dd925e5043918cde3acf1addf223" candidate_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" is_pre_election: true
I20260928 06:43:23.336599 24362 raft_consensus.cc:2454] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54aab4f020e6407e8a143ce6eb56d9bc in term 0.
I20260928 06:43:23.336618 24630 raft_consensus.cc:2454] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54aab4f020e6407e8a143ce6eb56d9bc in term 0.
I20260928 06:43:23.337066 24432 leader_election.cc:302] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54aab4f020e6407e8a143ce6eb56d9bc, e799d9ba35154838b05533257e3de6cc; no voters:
I20260928 06:43:23.337317 24701 raft_consensus.cc:2781] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:43:23.337405 24701 raft_consensus.cc:482] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:43:23.337453 24701 raft_consensus.cc:3037] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:23.338557 24701 raft_consensus.cc:504] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:23.338894 24701 leader_election.cc:288] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [CANDIDATE]: Term 1 election: Requested vote from peers 18ca0400c67b46e3aa45e0e4f355cc0f (127.8.151.193:33611), e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671)
I20260928 06:43:23.339304 24362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0551dd925e5043918cde3acf1addf223" candidate_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f"
I20260928 06:43:23.339324 24630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0551dd925e5043918cde3acf1addf223" candidate_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e799d9ba35154838b05533257e3de6cc"
I20260928 06:43:23.339418 24362 raft_consensus.cc:3037] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:23.339453 24630 raft_consensus.cc:3037] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:23.340422 24362 raft_consensus.cc:2454] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54aab4f020e6407e8a143ce6eb56d9bc in term 1.
I20260928 06:43:23.340682 24432 leader_election.cc:302] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18ca0400c67b46e3aa45e0e4f355cc0f, 54aab4f020e6407e8a143ce6eb56d9bc; no voters:
I20260928 06:43:23.340715 24630 raft_consensus.cc:2454] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54aab4f020e6407e8a143ce6eb56d9bc in term 1.
I20260928 06:43:23.340839 24701 raft_consensus.cc:2781] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:43:23.341161 24701 raft_consensus.cc:686] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 LEADER]: Becoming Leader. State: Replica: 54aab4f020e6407e8a143ce6eb56d9bc, State: Running, Role: LEADER
I20260928 06:43:23.341501 24701 consensus_queue.cc:237] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:23.344731 24217 catalog_manager.cc:5495] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 54aab4f020e6407e8a143ce6eb56d9bc (127.8.151.194). New cstate: current_term: 1 leader_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } health_report { overall_health: HEALTHY } } }
W20260928 06:43:23.348382 24677 tablet.cc:2369] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:43:23.360502 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.356908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.8.151.194:45037)
0928 06:43:23.357034 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:23.357038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:23.357050 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:43:23.357126 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:23.357128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:23.357184 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:23.357281 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:43:23.357541 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.358045 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.358901 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:23.359123 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:23.359245 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:23.359549 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:43:23.360079 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:43:23.360214 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:23.360255 (+ 41us) server_negotiation.cc:299] Negotiation successful
0928 06:43:23.360324 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:43:23.366930 24362 raft_consensus.cc:1261] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Refusing update from remote peer 54aab4f020e6407e8a143ce6eb56d9bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:43:23.366973 24630 raft_consensus.cc:1261] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc [term 1 FOLLOWER]: Refusing update from remote peer 54aab4f020e6407e8a143ce6eb56d9bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:43:23.367693 24701 consensus_queue.cc:1035] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:43:23.367981 24703 consensus_queue.cc:1035] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:43:23.375751 24707 mvcc.cc:205] Tried to move back new op lower bound from 7334206682583564288 to 7334206682489454592. Current Snapshot: MvccSnapshot[applied={T|T < 7334206682583564288}]
I20260928 06:43:23.376305 24706 mvcc.cc:205] Tried to move back new op lower bound from 7334206682583564288 to 7334206682489454592. Current Snapshot: MvccSnapshot[applied={T|T < 7334206682583564288}]
I20260928 06:43:23.385071 24217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:43:23.385251 24217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:43:23.387828 24217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Sending DeleteTablet for 3 replicas of tablet 0551dd925e5043918cde3acf1addf223
I20260928 06:43:23.388638 24610 tablet_service.cc:1514] Processing DeleteTablet for tablet 0551dd925e5043918cde3acf1addf223 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:23 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:46726
I20260928 06:43:23.388685 24342 tablet_service.cc:1514] Processing DeleteTablet for tablet 0551dd925e5043918cde3acf1addf223 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:23 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:43368
I20260928 06:43:23.388972 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 24279
I20260928 06:43:23.395350 24476 tablet_service.cc:1514] Processing DeleteTablet for tablet 0551dd925e5043918cde3acf1addf223 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:23 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:40838
I20260928 06:43:23.395439 24715 tablet_replica.cc:330] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: stopping tablet replica
I20260928 06:43:23.395818 24715 raft_consensus.cc:2229] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:43:23.395865 24717 tablet_replica.cc:330] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: stopping tablet replica
I20260928 06:43:23.396167 24715 raft_consensus.cc:2258] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:43:23.396174 24717 raft_consensus.cc:2229] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:43:23.396569 24717 raft_consensus.cc:2258] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:43:23.397783 24715 ts_tablet_manager.cc:1905] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:43:23.398030 24717 ts_tablet_manager.cc:1905] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:43:23.400525 24715 ts_tablet_manager.cc:1918] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:43:23.400627 24715 log.cc:1196] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/0551dd925e5043918cde3acf1addf223
I20260928 06:43:23.400795 24717 ts_tablet_manager.cc:1918] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:43:23.400900 24717 log.cc:1196] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/0551dd925e5043918cde3acf1addf223
I20260928 06:43:23.400947 24715 ts_tablet_manager.cc:1939] T 0551dd925e5043918cde3acf1addf223 P e799d9ba35154838b05533257e3de6cc: Deleting consensus metadata
I20260928 06:43:23.401216 24717 ts_tablet_manager.cc:1939] T 0551dd925e5043918cde3acf1addf223 P 54aab4f020e6407e8a143ce6eb56d9bc: Deleting consensus metadata
I20260928 06:43:23.402256 24204 catalog_manager.cc:4841] TS e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671): tablet 0551dd925e5043918cde3acf1addf223 (table test-table [id=810a51e49c3947b3828e5bcd61445e7d]) successfully deleted
I20260928 06:43:23.402297 24203 catalog_manager.cc:4841] TS 54aab4f020e6407e8a143ce6eb56d9bc (127.8.151.194:45037): tablet 0551dd925e5043918cde3acf1addf223 (table test-table [id=810a51e49c3947b3828e5bcd61445e7d]) successfully deleted
W20260928 06:43:23.407364 24204 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:33611 (error 108)
I20260928 06:43:23.407577 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 24413
I20260928 06:43:23.408962 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:23.408620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:33611 (local address 127.0.0.1:43380)
0928 06:43:23.408803 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:23.408869 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:33611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260928 06:43:23.409132 24204 catalog_manager.cc:4584] TS 18ca0400c67b46e3aa45e0e4f355cc0f (127.8.151.193:33611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0551dd925e5043918cde3acf1addf223: Network error: Client connection negotiation failed: client connection to 127.8.151.193:33611: connect: Connection refused (error 111)
I20260928 06:43:23.415598 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 24547
I20260928 06:43:23.420218 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 24185
I20260928 06:43:43.425930 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39155
--webserver_interface=127.8.151.254
--webserver_port=41135
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:39155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:43:43.535936 24719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:43.536186 24719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:43.536224 24719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:43.540221 24719 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 06:43:43.540278 24719 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 06:43:43.540292 24719 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 06:43:43.540304 24719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:43:43.540321 24719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:43.540337 24719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:43.540359 24719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:43:43.540386 24719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:43:43.545634 24719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:39155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:43.546656 24719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:43.547689 24719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:43.560433 24728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:43.560514 24719 server_base.cc:1034] running on GCE node
W20260928 06:43:43.560451 24725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:43.560724 24726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:43.561568 24719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:43.563012 24719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:43.564209 24719 hybrid_clock.cc:648] HybridClock initialized: now 1790577823564174 us; error 42 us; skew 500 ppm
Sep 28 06:43:43 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577823, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:43.572636 24719 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:43:43.573868 24719 webserver.cc:460] Webserver started at http://127.8.151.254:41135/ using document root <none> and password file <none>
I20260928 06:43:43.574589 24719 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:43.574692 24719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:43.579768 24719 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 06:43:43.582520 24735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:43.584234 24719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:43:43.584364 24719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "3f51f5b0dc6d435f8dee647b2a20372a"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "72dd8593cf991088a6206dc56286dda4"
server_key_iv: "245dd6f5fff0e15153ff14e10bfcbddb"
server_key_version: "encryptionkey@0"
I20260928 06:43:43.585108 24719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:43.612723 24719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:43.616490 24719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:43.616698 24719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:43.625267 24719 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:39155
I20260928 06:43:43.625540 24787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:39155 every 8 connection(s)
I20260928 06:43:43.626739 24719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:43:43.631314 24788 sys_catalog.cc:263] Verifying existing consensus state
I20260928 06:43:43.633008 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 24719
I20260928 06:43:43.633188 24788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Bootstrap starting.
I20260928 06:43:43.633478 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:33611
--local_ip_for_outbound_sockets=127.8.151.193
--tserver_master_addrs=127.8.151.254:39155
--webserver_port=39103
--webserver_interface=127.8.151.193
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:43:43.650382 24788 log.cc:824] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:43.659477 24788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:43:43.659869 24788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Bootstrap complete.
I20260928 06:43:43.663797 24788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:43:43.664211 24788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f51f5b0dc6d435f8dee647b2a20372a, State: Initialized, Role: FOLLOWER
I20260928 06:43:43.664739 24788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:43:43.664865 24788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:43:43.664952 24788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:43:43.665076 24788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 1 FOLLOWER]: Advancing to term 2
I20260928 06:43:43.666944 24788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:43:43.667308 24788 leader_election.cc:302] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f51f5b0dc6d435f8dee647b2a20372a; no voters:
I20260928 06:43:43.667630 24788 leader_election.cc:288] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 06:43:43.667729 24792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 2 FOLLOWER]: Leader election won for term 2
I20260928 06:43:43.668116 24792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 2 LEADER]: Becoming Leader. State: Replica: 3f51f5b0dc6d435f8dee647b2a20372a, State: Running, Role: LEADER
I20260928 06:43:43.668560 24792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:43:43.668921 24788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:43:43.669557 24792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } } }
I20260928 06:43:43.669764 24792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:43.670002 24792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f51f5b0dc6d435f8dee647b2a20372a. Latest consensus state: current_term: 2 leader_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } } }
I20260928 06:43:43.670142 24792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: This master's current role is: LEADER
I20260928 06:43:43.670426 24800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:43:43.676855 24800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:43:43.677825 24800 catalog_manager.cc:1259] Loaded cluster ID: 8b5a88b2ef2b447985f35f88716c6942
I20260928 06:43:43.677950 24800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:43:43.680675 24800 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:43:43.681432 24800 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Loaded TSK: 0
I20260928 06:43:43.694413 24800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Generated new TSK 1
I20260928 06:43:43.694769 24800 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 06:43:43.789021 24790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:43.789466 24790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:43.789534 24790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:43.795712 24790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:43.795802 24790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:43.795892 24790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:43:43.803880 24790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:33611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=39103
--tserver_master_addrs=127.8.151.254:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:43.805341 24790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:43.806768 24790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:43.819088 24816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:43.819118 24815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:43.819501 24790 server_base.cc:1034] running on GCE node
W20260928 06:43:43.819530 24818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:43.820020 24790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:43.820675 24790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:43.821862 24790 hybrid_clock.cc:648] HybridClock initialized: now 1790577823821837 us; error 40 us; skew 500 ppm
Sep 28 06:43:43 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577823, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:43.824795 24790 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:43:43.825706 24790 webserver.cc:460] Webserver started at http://127.8.151.193:39103/ using document root <none> and password file <none>
I20260928 06:43:43.826258 24790 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:43.826351 24790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:43.830222 24790 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:43:43.832382 24825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:43.833627 24790 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:43:43.833730 24790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "18ca0400c67b46e3aa45e0e4f355cc0f"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "e1fa75c648384d39a20a390f09a7fda1"
server_key_iv: "decd9a2064effe14422f08b3ea286632"
server_key_version: "encryptionkey@0"
I20260928 06:43:43.834249 24790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:43.856132 24790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:43.858835 24790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:43.858994 24790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:43.859586 24790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:43.860594 24832 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 06:43:43.865393 24790 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 06:43:43.865453 24790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260928 06:43:43.865502 24790 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 06:43:43.867100 24790 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 06:43:43.867167 24790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260928 06:43:43.867455 24832 tablet_bootstrap.cc:492] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Bootstrap starting.
I20260928 06:43:43.879345 24790 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:33611
I20260928 06:43:43.880311 24790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:43:43.881966 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 24790
I20260928 06:43:43.882421 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:45037
--local_ip_for_outbound_sockets=127.8.151.194
--tserver_master_addrs=127.8.151.254:39155
--webserver_port=39433
--webserver_interface=127.8.151.194
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:43:43.885404 24939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:33611 every 8 connection(s)
I20260928 06:43:43.889961 24832 log.cc:824] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Log is configured to *not* fsync() on all Append() calls
Sep 28 06:43:43 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577823, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:43.899566 24832 tablet_bootstrap.cc:492] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:43:43.900643 24832 tablet_bootstrap.cc:492] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Bootstrap complete.
I20260928 06:43:43.901939 24832 ts_tablet_manager.cc:1397] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Time spent bootstrapping tablet: real 0.035s user 0.019s sys 0.012s
I20260928 06:43:43.905074 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:43.883994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:47767 (local address 127.8.151.254:39155)
0928 06:43:43.884336 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:43.884354 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:43.885997 (+ 1643us) server_negotiation.cc:407] Connection header received
0928 06:43:43.888817 (+ 2820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:43.888836 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:43.889294 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:43.889682 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:43.891793 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:43.893366 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:43.895119 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:43.895885 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:43.896016 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:43.899299 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:43.899307 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:43.899322 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:43.899354 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:43.901349 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:43.901944 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:43.901950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:43.901956 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:43.902112 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:43.902499 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:43.902503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:43.902505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:43.902967 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:43.903092 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:43.903790 (+ 698us) server_negotiation.cc:299] Negotiation successful
0928 06:43:43.904122 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":104,"threads_started":1}
I20260928 06:43:43.904917 24832 raft_consensus.cc:348] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:43.906021 24832 raft_consensus.cc:729] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18ca0400c67b46e3aa45e0e4f355cc0f, State: Initialized, Role: FOLLOWER
I20260928 06:43:43.906610 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:43.882470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:47767)
0928 06:43:43.885797 (+ 3327us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:43.885853 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:43.886971 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:43.890386 (+ 3415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:43.890396 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:43.890875 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:43.891581 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:43.891603 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:43.893558 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:43.893562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:43.894959 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:43.894970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:43.896314 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:43.896318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:43.896505 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:43.897122 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:43.897143 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:43.899067 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:43.901480 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:43.901486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:43.901499 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:43.901818 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:43.902221 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:43.902224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:43.902226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:43.902384 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:43.903184 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:43.903190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:43.903460 (+ 270us) client_negotiation.cc:769] Sending connection context
0928 06:43:43.905347 (+ 1887us) client_negotiation.cc:240] Negotiation successful
0928 06:43:43.905670 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3107,"thread_start_us":127,"threads_started":1}
I20260928 06:43:43.906951 24832 consensus_queue.cc:260] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } } peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } }
I20260928 06:43:43.908145 24832 ts_tablet_manager.cc:1428] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260928 06:43:43.909211 24940 heartbeater.cc:347] Connected to a master server at 127.8.151.254:39155
I20260928 06:43:43.909431 24940 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:43.909973 24940 heartbeater.cc:511] Master 127.8.151.254:39155 requested a full tablet report, sending...
I20260928 06:43:43.912618 24752 ts_manager.cc:194] Registered new tserver with Master: 18ca0400c67b46e3aa45e0e4f355cc0f (127.8.151.193:33611)
I20260928 06:43:43.917675 24752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:47767
I20260928 06:43:43.920588 24940 heartbeater.cc:503] Master 127.8.151.254:39155 was elected leader, sending a full tablet report...
Sep 28 06:43:43 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577823, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:43:43.930464 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:43.916567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44994 (local address 127.8.151.193:33611)
0928 06:43:43.918636 (+ 2069us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:43.918649 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:43.918674 (+ 25us) server_negotiation.cc:407] Connection header received
0928 06:43:43.918747 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:43.918753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:43.918963 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:43.919981 (+ 1018us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:43.920145 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:43.921204 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:43.923405 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:43.923923 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:43.924031 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:43.926690 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:43.926701 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:43.926714 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:43.926741 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:43.928363 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:43.928842 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:43.928848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:43.928850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:43.928943 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:43.929288 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:43.929291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:43.929293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:43.929580 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:43.929746 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:43.930142 (+ 396us) server_negotiation.cc:299] Negotiation successful
0928 06:43:43.930293 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1963,"thread_start_us":110,"threads_started":1}
I20260928 06:43:43.930464 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:43.915803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:33611 (local address 127.0.0.1:44994)
0928 06:43:43.916254 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:43.916311 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:43.916451 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:43.919177 (+ 2726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:43.919183 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:43.919657 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:43.919820 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:43.919829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:43.922186 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:43.922189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:43.923255 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:43.923267 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:43.924076 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:43.924080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:43.924193 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:43.924792 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:43.924806 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:43.926550 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:43.928471 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:43.928481 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:43.928487 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:43.928728 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:43.929069 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:43.929072 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:43.929074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:43.929168 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:43.929742 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:43.929748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:43.929880 (+ 132us) client_negotiation.cc:769] Sending connection context
0928 06:43:43.930051 (+ 171us) client_negotiation.cc:240] Negotiation successful
0928 06:43:43.930268 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":105,"threads_started":1}
I20260928 06:43:43.931835 24862 tablet_service.cc:1514] Processing DeleteTablet for tablet 0551dd925e5043918cde3acf1addf223 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:23 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:44994
I20260928 06:43:43.933037 24949 tablet_replica.cc:330] stopping tablet replica
I20260928 06:43:43.933307 24949 raft_consensus.cc:2229] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:43:43.933542 24949 raft_consensus.cc:2258] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:43:43.934444 24949 ts_tablet_manager.cc:1905] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:43:43.937951 24949 ts_tablet_manager.cc:1918] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:43:43.938035 24949 log.cc:1196] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal/wals/0551dd925e5043918cde3acf1addf223
I20260928 06:43:43.938248 24949 ts_tablet_manager.cc:1939] T 0551dd925e5043918cde3acf1addf223 P 18ca0400c67b46e3aa45e0e4f355cc0f: Deleting consensus metadata
I20260928 06:43:43.940292 24739 catalog_manager.cc:4841] TS 18ca0400c67b46e3aa45e0e4f355cc0f (127.8.151.193:33611): tablet 0551dd925e5043918cde3acf1addf223 (table test-table [id=810a51e49c3947b3828e5bcd61445e7d]) successfully deleted
W20260928 06:43:44.009938 24943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:44.010212 24943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:44.010274 24943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:44.014417 24943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:44.014485 24943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:44.014571 24943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:43:44.019794 24943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:45037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=39433
--tserver_master_addrs=127.8.151.254:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:44.020782 24943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:44.021749 24943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:44.031306 24957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:44.031318 24954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:44.031543 24955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:44.031508 24943 server_base.cc:1034] running on GCE node
I20260928 06:43:44.031980 24943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:44.032719 24943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:44.033903 24943 hybrid_clock.cc:648] HybridClock initialized: now 1790577824033855 us; error 60 us; skew 500 ppm
Sep 28 06:43:44 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577824, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:44.036978 24943 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:43:44.037952 24943 webserver.cc:460] Webserver started at http://127.8.151.194:39433/ using document root <none> and password file <none>
I20260928 06:43:44.038635 24943 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:44.038736 24943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:44.043463 24943 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:43:44.045905 24964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:44.047286 24943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:43:44.047390 24943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "54aab4f020e6407e8a143ce6eb56d9bc"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "549089452f305a42073c7ebfd3c45fd7"
server_key_iv: "82e94efe59702be7324d588581a4c883"
server_key_version: "encryptionkey@0"
I20260928 06:43:44.048054 24943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:44.074119 24943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:44.077422 24943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:44.077617 24943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:44.078330 24943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:44.079416 24943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:44.079474 24943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:44.079536 24943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:44.079573 24943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:44.088601 24943 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:45037
I20260928 06:43:44.088663 25077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:45037 every 8 connection(s)
I20260928 06:43:44.089634 24943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:43:44.091451 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 24943
I20260928 06:43:44.091885 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:44671
--local_ip_for_outbound_sockets=127.8.151.195
--tserver_master_addrs=127.8.151.254:39155
--webserver_port=39525
--webserver_interface=127.8.151.195
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 06:43:44 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577824, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:44.102869 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.091481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:37423 (local address 127.8.151.254:39155)
0928 06:43:44.091641 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:44.091645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:44.092794 (+ 1149us) server_negotiation.cc:407] Connection header received
0928 06:43:44.093893 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:44.093897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:44.093957 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:44.094052 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:44.095378 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.095913 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.097063 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.097298 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.097388 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.099687 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:44.099691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:44.099706 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:44.099729 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:44.101021 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:44.101474 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:44.101477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:44.101478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:44.101558 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:44.101854 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:44.101857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:44.101858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:44.102024 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:44.102109 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:44.102578 (+ 469us) server_negotiation.cc:299] Negotiation successful
0928 06:43:44.102704 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260928 06:43:44.103861 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.091930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:37423)
0928 06:43:44.092650 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:44.092701 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:44.093652 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:44.094261 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:44.094270 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:44.094699 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:44.095236 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.095253 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.096032 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.096035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.096941 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.096950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.097435 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.097439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.097616 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.098144 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:44.098160 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:44.099524 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:44.101128 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:44.101134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:44.101147 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:44.101381 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:44.101638 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:44.101641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:44.101643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:44.101768 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:44.102122 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:44.102128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:44.102353 (+ 225us) client_negotiation.cc:769] Sending connection context
0928 06:43:44.102510 (+ 157us) client_negotiation.cc:240] Negotiation successful
0928 06:43:44.102811 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":133,"threads_started":1}
I20260928 06:43:44.104961 25078 heartbeater.cc:347] Connected to a master server at 127.8.151.254:39155
I20260928 06:43:44.105218 25078 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:44.105670 25078 heartbeater.cc:511] Master 127.8.151.254:39155 requested a full tablet report, sending...
I20260928 06:43:44.106627 24752 ts_manager.cc:194] Registered new tserver with Master: 54aab4f020e6407e8a143ce6eb56d9bc (127.8.151.194:45037)
I20260928 06:43:44.107190 24752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:37423
W20260928 06:43:44.218933 25081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:43:44.219249 25081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:43:44.219303 25081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:43:44.223440 25081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:43:44.223496 25081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:43:44.223560 25081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:43:44.229000 25081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:44671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=39525
--tserver_master_addrs=127.8.151.254:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:43:44.230024 25081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:43:44.231009 25081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:43:44.240497 25086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:44.240610 25087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:43:44.240504 25089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:43:44.240825 25081 server_base.cc:1034] running on GCE node
I20260928 06:43:44.241339 25081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:43:44.241963 25081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:43:44.243129 25081 hybrid_clock.cc:648] HybridClock initialized: now 1790577824243103 us; error 44 us; skew 500 ppm
Sep 28 06:43:44 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577824, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:43:44.245781 25081 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:43:44.246661 25081 webserver.cc:460] Webserver started at http://127.8.151.195:39525/ using document root <none> and password file <none>
I20260928 06:43:44.247249 25081 fs_manager.cc:359] Metadata directory not provided
I20260928 06:43:44.247335 25081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:43:44.251523 25081 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:43:44.253695 25096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:44.254968 25081 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:43:44.255084 25081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "e799d9ba35154838b05533257e3de6cc"
format_stamp: "Formatted at 2026-09-28 06:43:23 on dist-test-slave-gtr8"
server_key: "4d6869c7f7ff7842de004974f45e9556"
server_key_iv: "96a437a92370187a2ef4fe4e6aaf8a60"
server_key_version: "encryptionkey@0"
I20260928 06:43:44.255659 25081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:43:44.278908 25081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:43:44.282218 25081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:43:44.282403 25081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:43:44.283023 25081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:43:44.283874 25081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:43:44.283923 25081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:44.283977 25081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:43:44.284003 25081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:43:44.293823 25081 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:44671
I20260928 06:43:44.293841 25209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:44671 every 8 connection(s)
I20260928 06:43:44.294836 25081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:43:44.299372 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 25081
Sep 28 06:43:44 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577824, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:43:44.309083 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.299962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.8.151.254:39155)
0928 06:43:44.300213 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:44.300217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:44.300477 (+ 260us) server_negotiation.cc:407] Connection header received
0928 06:43:44.300592 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:44.300595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:44.300637 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:44.300708 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:44.301578 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.302267 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.303364 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.303599 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.303686 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.305045 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:44.305049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:44.305052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:44.305074 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:44.306442 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:44.307086 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:44.307089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:44.307091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:44.307231 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:44.307736 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:44.307739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:44.307740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:44.307948 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:44.308138 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:44.308785 (+ 647us) server_negotiation.cc:299] Negotiation successful
0928 06:43:44.308905 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
I20260928 06:43:44.310505 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.296638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:37839 (local address 127.8.151.254:39155)
0928 06:43:44.296785 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:44.296790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:44.297935 (+ 1145us) server_negotiation.cc:407] Connection header received
0928 06:43:44.299009 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:44.299013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:44.299089 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:44.299218 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:44.300890 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.301646 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.302990 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.303313 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.303404 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.306654 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:44.306657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:44.306660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:44.306684 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:44.307973 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:44.308611 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:44.308615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:44.308617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:44.308706 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:44.309255 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:44.309258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:44.309260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:44.309456 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:44.309546 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:44.310257 (+ 711us) server_negotiation.cc:299] Negotiation successful
0928 06:43:44.310369 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 06:43:44.311636 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.297038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:37839)
0928 06:43:44.297770 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:44.297826 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:44.298782 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:44.299380 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:44.299389 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:44.299871 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:43:44.300752 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.300770 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.301762 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.301765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.302868 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.302880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.303584 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.303588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.303792 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.304476 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:43:44.304495 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:43:44.306434 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:43:44.308113 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:44.308119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:44.308135 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:44.308485 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:44.308823 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:43:44.308826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:43:44.308828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:43:44.309031 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:43:44.309617 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:43:44.309623 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:43:44.309985 (+ 362us) client_negotiation.cc:769] Sending connection context
0928 06:43:44.310214 (+ 229us) client_negotiation.cc:240] Negotiation successful
0928 06:43:44.310704 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":121,"threads_started":1}
I20260928 06:43:44.312711 25210 heartbeater.cc:347] Connected to a master server at 127.8.151.254:39155
I20260928 06:43:44.312937 25210 heartbeater.cc:464] Registering TS with master...
I20260928 06:43:44.313396 25210 heartbeater.cc:511] Master 127.8.151.254:39155 requested a full tablet report, sending...
I20260928 06:43:44.314368 24752 ts_manager.cc:194] Registered new tserver with Master: e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671)
I20260928 06:43:44.314894 24752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:37839
I20260928 06:43:44.322649 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:43:44.326689 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.323239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46492 (local address 127.8.151.254:39155)
0928 06:43:44.323356 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:44.323361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:44.323594 (+ 233us) server_negotiation.cc:407] Connection header received
0928 06:43:44.323761 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:44.323764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:44.323862 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:44.323950 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:43:44.324190 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.324690 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.325568 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.325780 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.325863 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.326149 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:43:44.326290 (+ 141us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0928 06:43:44.326541 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46492: token expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260928 06:43:44.326756 24944 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46492: token expired
W20260928 06:43:44.327565 24680 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.151.254:39155)
I20260928 06:43:44.334619 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.327898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46496 (local address 127.8.151.254:39155)
0928 06:43:44.327995 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:44.327999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:44.328078 (+ 79us) server_negotiation.cc:407] Connection header received
0928 06:43:44.328189 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:44.328191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:44.328230 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:44.328288 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:43:44.328956 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.329535 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.330300 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.330513 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.330594 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.331846 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:43:44.331849 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:43:44.331852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:43:44.331877 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:43:44.333126 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:44.333499 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:44.333502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:44.333504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:44.333600 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:43:44.333841 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:43:44.333844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:43:44.333846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:43:44.334028 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:43:44.334143 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:44.334368 (+ 225us) server_negotiation.cc:299] Negotiation successful
0928 06:43:44.334474 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 06:43:44.337108 24752 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:43:44.340401 24752 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:43:44.348583 24862 tablet_service.cc:1467] Processing CreateTablet for tablet 33704e80a9bd4f0090c03694cafb2629 (DEFAULT_TABLE table=test-table [id=b13c0ee062074d3bbb3434766fdd9008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:44.349248 24862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33704e80a9bd4f0090c03694cafb2629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:44.352192 24832 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Bootstrap starting.
I20260928 06:43:44.353101 24832 tablet_bootstrap.cc:654] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:44.353411 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.347054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:44671 (local address 127.0.0.1:42612)
0928 06:43:44.347257 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:44.347272 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:44.347346 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:44.348360 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:44.348363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:44.348378 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:44.348480 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.348486 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.349757 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.349760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.350891 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.350902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.352960 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.352963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.353064 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.353122 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 06:43:44.353173 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 06:43:44.353258 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260928 06:43:44.353742 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.347331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:45037 (local address 127.0.0.1:36930)
0928 06:43:44.348103 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:44.348115 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:44.348201 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:44.348855 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:44.348858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:44.348874 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:44.348960 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.348967 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.350457 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.350461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.351566 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.351576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.353277 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.353281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.353381 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.353406 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 06:43:44.353475 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 06:43:44.353545 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":106,"threads_started":1}
I20260928 06:43:44.354317 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.347513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36930 (local address 127.8.151.194:45037)
0928 06:43:44.347827 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:44.347838 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:44.348189 (+ 351us) server_negotiation.cc:407] Connection header received
0928 06:43:44.348307 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:44.348312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:44.348491 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:44.348644 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:44.349188 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.350331 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.351866 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.352951 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.353051 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.353208 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:44.354076 (+ 868us) server_negotiation.cc:299] Negotiation successful
0928 06:43:44.354181 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":124,"threads_started":1}
I20260928 06:43:44.354517 24832 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: No bootstrap required, opened a new log
I20260928 06:43:44.354586 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.347536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42612 (local address 127.8.151.195:44671)
0928 06:43:44.347845 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:44.347859 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:44.347883 (+ 24us) server_negotiation.cc:407] Connection header received
0928 06:43:44.347962 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:44.347968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:44.348217 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:44.348378 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:44.348585 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.349644 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.351036 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.352827 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.354111 (+ 1284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.354278 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:44.354335 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 06:43:44.354432 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":135,"threads_started":1}
I20260928 06:43:44.354609 24832 ts_tablet_manager.cc:1397] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 06:43:44.355063 24832 raft_consensus.cc:348] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:43:44.355211 24832 raft_consensus.cc:374] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:44.355250 24832 raft_consensus.cc:729] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18ca0400c67b46e3aa45e0e4f355cc0f, State: Initialized, Role: FOLLOWER
I20260928 06:43:44.355377 24832 consensus_queue.cc:260] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:43:44.356128 24832 ts_tablet_manager.cc:1428] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 06:43:44.357406 25144 tablet_service.cc:1467] Processing CreateTablet for tablet 33704e80a9bd4f0090c03694cafb2629 (DEFAULT_TABLE table=test-table [id=b13c0ee062074d3bbb3434766fdd9008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:44.357406 25012 tablet_service.cc:1467] Processing CreateTablet for tablet 33704e80a9bd4f0090c03694cafb2629 (DEFAULT_TABLE table=test-table [id=b13c0ee062074d3bbb3434766fdd9008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:43:44.358529 25144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33704e80a9bd4f0090c03694cafb2629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:44.358771 25012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33704e80a9bd4f0090c03694cafb2629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:43:44.367352 25223 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Bootstrap starting.
I20260928 06:43:44.367642 25224 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Bootstrap starting.
I20260928 06:43:44.369819 25223 tablet_bootstrap.cc:654] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:44.370193 25224 tablet_bootstrap.cc:654] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Neither blocks nor log segments found. Creating new log.
I20260928 06:43:44.370587 25223 log.cc:824] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:44.371007 25224 log.cc:824] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Log is configured to *not* fsync() on all Append() calls
I20260928 06:43:44.372892 25223 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: No bootstrap required, opened a new log
I20260928 06:43:44.372892 25224 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: No bootstrap required, opened a new log
I20260928 06:43:44.373081 25224 ts_tablet_manager.cc:1397] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 06:43:44.373121 25223 ts_tablet_manager.cc:1397] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:43:44.376955 25224 raft_consensus.cc:348] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:43:44.376955 25223 raft_consensus.cc:348] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:43:44.377252 25224 raft_consensus.cc:374] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:44.377313 25224 raft_consensus.cc:729] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e799d9ba35154838b05533257e3de6cc, State: Initialized, Role: FOLLOWER
I20260928 06:43:44.377341 25223 raft_consensus.cc:374] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:43:44.377468 25223 raft_consensus.cc:729] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54aab4f020e6407e8a143ce6eb56d9bc, State: Initialized, Role: FOLLOWER
I20260928 06:43:44.377839 25224 consensus_queue.cc:260] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:43:44.377871 25223 consensus_queue.cc:260] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:43:44.378567 25210 heartbeater.cc:503] Master 127.8.151.254:39155 was elected leader, sending a full tablet report...
I20260928 06:43:44.378567 25078 heartbeater.cc:503] Master 127.8.151.254:39155 was elected leader, sending a full tablet report...
I20260928 06:43:44.378969 25223 ts_tablet_manager.cc:1428] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:43:44.379061 25224 ts_tablet_manager.cc:1428] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260928 06:43:44.386201 24941 tablet.cc:2369] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:43:44.443030 25227 raft_consensus.cc:482] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:43:44.443290 25227 raft_consensus.cc:504] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:43:44.444765 25227 leader_election.cc:288] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54aab4f020e6407e8a143ce6eb56d9bc (127.8.151.194:45037), 18ca0400c67b46e3aa45e0e4f355cc0f (127.8.151.193:33611)
I20260928 06:43:44.449131 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.445148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:60837 (local address 127.8.151.194:45037)
0928 06:43:44.445268 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:44.445272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:44.445288 (+ 16us) server_negotiation.cc:407] Connection header received
0928 06:43:44.445346 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:44.445349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:44.445399 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:44.445491 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:44.445899 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.446609 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.448014 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.448582 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.448697 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.448747 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:44.448931 (+ 184us) server_negotiation.cc:299] Negotiation successful
0928 06:43:44.449005 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:43:44.449134 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.444962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:45037 (local address 127.8.151.195:60837)
0928 06:43:44.445140 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:44.445154 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:44.445244 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:44.445630 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:44.445633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:44.445651 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:44.445780 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.445787 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.446737 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.446740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.447881 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.447893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.448702 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.448705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.448808 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.448832 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 06:43:44.448913 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 06:43:44.448995 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":1,"spinlock_wait_cycles":2432}
I20260928 06:43:44.449388 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.445173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:44411 (local address 127.8.151.193:33611)
0928 06:43:44.445441 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:44.445446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:44.445666 (+ 220us) server_negotiation.cc:407] Connection header received
0928 06:43:44.445780 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:44.445782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:44.445835 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:44.445947 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:43:44.446215 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.446842 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.447928 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.448575 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.448697 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.448839 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:44.449187 (+ 348us) server_negotiation.cc:299] Negotiation successful
0928 06:43:44.449259 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":88,"threads_started":1}
I20260928 06:43:44.449579 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.444962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:33611 (local address 127.8.151.195:44411)
0928 06:43:44.445579 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:44.445592 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:43:44.445698 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:43:44.445961 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:43:44.445963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:43:44.445980 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:43:44.446094 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.446100 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.446954 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.446958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.447812 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:43:44.447821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.448824 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.448827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:43:44.448927 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.448958 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 06:43:44.449022 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 06:43:44.449084 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":349,"spinlock_wait_cycles":2176,"thread_start_us":95,"threads_started":1}
I20260928 06:43:44.449987 25032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33704e80a9bd4f0090c03694cafb2629" candidate_uuid: "e799d9ba35154838b05533257e3de6cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" is_pre_election: true
I20260928 06:43:44.450166 24893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33704e80a9bd4f0090c03694cafb2629" candidate_uuid: "e799d9ba35154838b05533257e3de6cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" is_pre_election: true
I20260928 06:43:44.450305 25032 raft_consensus.cc:2454] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e799d9ba35154838b05533257e3de6cc in term 0.
I20260928 06:43:44.450443 24893 raft_consensus.cc:2454] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e799d9ba35154838b05533257e3de6cc in term 0.
I20260928 06:43:44.450911 25100 leader_election.cc:302] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18ca0400c67b46e3aa45e0e4f355cc0f, e799d9ba35154838b05533257e3de6cc; no voters:
I20260928 06:43:44.451423 25227 raft_consensus.cc:2781] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:43:44.451525 25227 raft_consensus.cc:482] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:43:44.451576 25227 raft_consensus.cc:3037] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:44.452845 25227 raft_consensus.cc:504] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:43:44.453220 25227 leader_election.cc:288] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [CANDIDATE]: Term 1 election: Requested vote from peers 54aab4f020e6407e8a143ce6eb56d9bc (127.8.151.194:45037), 18ca0400c67b46e3aa45e0e4f355cc0f (127.8.151.193:33611)
I20260928 06:43:44.453579 24893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33704e80a9bd4f0090c03694cafb2629" candidate_uuid: "e799d9ba35154838b05533257e3de6cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f"
I20260928 06:43:44.453590 25032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33704e80a9bd4f0090c03694cafb2629" candidate_uuid: "e799d9ba35154838b05533257e3de6cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54aab4f020e6407e8a143ce6eb56d9bc"
I20260928 06:43:44.453713 25032 raft_consensus.cc:3037] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:44.453735 24893 raft_consensus.cc:3037] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:43:44.454766 25032 raft_consensus.cc:2454] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e799d9ba35154838b05533257e3de6cc in term 1.
I20260928 06:43:44.454933 24893 raft_consensus.cc:2454] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e799d9ba35154838b05533257e3de6cc in term 1.
I20260928 06:43:44.455052 25099 leader_election.cc:302] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54aab4f020e6407e8a143ce6eb56d9bc, e799d9ba35154838b05533257e3de6cc; no voters:
I20260928 06:43:44.455279 25227 raft_consensus.cc:2781] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:43:44.455559 25227 raft_consensus.cc:686] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 1 LEADER]: Becoming Leader. State: Replica: e799d9ba35154838b05533257e3de6cc, State: Running, Role: LEADER
I20260928 06:43:44.455931 25227 consensus_queue.cc:237] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:43:44.459663 24752 catalog_manager.cc:5495] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc reported cstate change: term changed from 0 to 1, leader changed from <none> to e799d9ba35154838b05533257e3de6cc (127.8.151.195). New cstate: current_term: 1 leader_uuid: "e799d9ba35154838b05533257e3de6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } health_report { overall_health: UNKNOWN } } }
I20260928 06:43:44.482879 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.478874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42618 (local address 127.8.151.195:44671)
0928 06:43:44.479266 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0928 06:43:44.479269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:43:44.479282 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:43:44.479365 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:43:44.479367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:43:44.479422 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:43:44.479505 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:43:44.479796 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.480278 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.481420 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:43:44.481619 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:43:44.481710 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:43:44.482050 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:43:44.482481 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:43:44.482567 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 06:43:44.482681 (+ 114us) server_negotiation.cc:299] Negotiation successful
0928 06:43:44.482738 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260928 06:43:44.489423 24893 raft_consensus.cc:1261] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Refusing update from remote peer e799d9ba35154838b05533257e3de6cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:43:44.489426 25032 raft_consensus.cc:1261] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Refusing update from remote peer e799d9ba35154838b05533257e3de6cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:43:44.490247 25227 consensus_queue.cc:1035] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:43:44.490442 25231 consensus_queue.cc:1035] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:43:44.497068 25235 mvcc.cc:205] Tried to move back new op lower bound from 7334206769101123584 to 7334206768974704640. Current Snapshot: MvccSnapshot[applied={T|T < 7334206769101123584}]
I20260928 06:43:44.502509 25236 mvcc.cc:205] Tried to move back new op lower bound from 7334206769101123584 to 7334206768974704640. Current Snapshot: MvccSnapshot[applied={T|T < 7334206769101123584}]
I20260928 06:43:44.504628 24752 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:43:44.504772 24752 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:43:44.506735 24752 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Sending DeleteTablet for 3 replicas of tablet 33704e80a9bd4f0090c03694cafb2629
I20260928 06:43:44.507525 25144 tablet_service.cc:1514] Processing DeleteTablet for tablet 33704e80a9bd4f0090c03694cafb2629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:44 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:42612
I20260928 06:43:44.507701 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 24790
I20260928 06:43:44.507977 25012 tablet_service.cc:1514] Processing DeleteTablet for tablet 33704e80a9bd4f0090c03694cafb2629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:43:44 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:36930
I20260928 06:43:44.508396 25245 tablet_replica.cc:330] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: stopping tablet replica
I20260928 06:43:44.508802 25244 tablet_replica.cc:330] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: stopping tablet replica
I20260928 06:43:44.508821 25245 raft_consensus.cc:2229] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:43:44.509157 25244 raft_consensus.cc:2229] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:43:44.509176 25245 raft_consensus.cc:2258] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:43:44.509558 25244 raft_consensus.cc:2258] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:43:44.511030 25244 ts_tablet_manager.cc:1905] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:43:44.513736 25244 ts_tablet_manager.cc:1918] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:43:44.513890 25244 log.cc:1196] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/33704e80a9bd4f0090c03694cafb2629
I20260928 06:43:44.514271 25244 ts_tablet_manager.cc:1939] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Deleting consensus metadata
W20260928 06:43:44.514575 24739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:33611 (error 108)
I20260928 06:43:44.516289 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 24943
I20260928 06:43:44.516486 25245 ts_tablet_manager.cc:1905] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:43:44.517412 24739 catalog_manager.cc:4841] TS e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671): tablet 33704e80a9bd4f0090c03694cafb2629 (table test-table [id=b13c0ee062074d3bbb3434766fdd9008]) successfully deleted
I20260928 06:43:44.522657 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 25081
I20260928 06:43:44.523422 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.522799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:33611 (local address 127.0.0.1:45008)
0928 06:43:44.523259 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:44.523333 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:33611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378}
W20260928 06:43:44.523738 24739 catalog_manager.cc:4584] TS 18ca0400c67b46e3aa45e0e4f355cc0f (127.8.151.193:33611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33704e80a9bd4f0090c03694cafb2629: Network error: Client connection negotiation failed: client connection to 127.8.151.193:33611: connect: Connection refused (error 111)
I20260928 06:43:44.525086 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:43:44.524754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:45037 (local address 127.0.0.1:36932)
0928 06:43:44.524952 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 06:43:44.525000 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.194:45037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260928 06:43:44.528587 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 24719
I20260928 06:44:04.534010 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39155
--webserver_interface=127.8.151.254
--webserver_port=41135
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:39155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:04.648689 25247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:04.649060 25247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:04.649194 25247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:04.654650 25247 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 06:44:04.654728 25247 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 06:44:04.654754 25247 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 06:44:04.654778 25247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:04.654811 25247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:04.654839 25247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:04.654870 25247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:04.654901 25247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:04.662147 25247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:39155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:04.663677 25247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:04.665041 25247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:04.676116 25253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:04.676535 25252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:04.676725 25255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:04.677538 25247 server_base.cc:1034] running on GCE node
I20260928 06:44:04.678318 25247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:04.679925 25247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:04.681255 25247 hybrid_clock.cc:648] HybridClock initialized: now 1790577844681220 us; error 52 us; skew 500 ppm
Sep 28 06:44:04 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577844, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:04.685184 25247 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:04.686408 25247 webserver.cc:460] Webserver started at http://127.8.151.254:41135/ using document root <none> and password file <none>
I20260928 06:44:04.687192 25247 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:04.687286 25247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:04.692588 25247 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260928 06:44:04.695358 25262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:04.697057 25247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:44:04.697201 25247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "3f51f5b0dc6d435f8dee647b2a20372a"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "72dd8593cf991088a6206dc56286dda4"
server_key_iv: "245dd6f5fff0e15153ff14e10bfcbddb"
server_key_version: "encryptionkey@0"
I20260928 06:44:04.697914 25247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:04.736395 25247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:04.740037 25247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:04.740197 25247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:04.747538 25247 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:39155
I20260928 06:44:04.747818 25314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:39155 every 8 connection(s)
I20260928 06:44:04.748672 25247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:04.751427 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 25247
I20260928 06:44:04.751875 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:33611
--local_ip_for_outbound_sockets=127.8.151.193
--tserver_master_addrs=127.8.151.254:39155
--webserver_port=39103
--webserver_interface=127.8.151.193
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:44:04.753235 25315 sys_catalog.cc:263] Verifying existing consensus state
I20260928 06:44:04.755327 25315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Bootstrap starting.
I20260928 06:44:04.771795 25315 log.cc:824] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:04.782857 25315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:44:04.783263 25315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Bootstrap complete.
I20260928 06:44:04.786993 25315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:44:04.787439 25315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f51f5b0dc6d435f8dee647b2a20372a, State: Initialized, Role: FOLLOWER
I20260928 06:44:04.787940 25315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:44:04.788061 25315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:04.788125 25315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:04.788215 25315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 2 FOLLOWER]: Advancing to term 3
I20260928 06:44:04.790012 25315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:44:04.790331 25315 leader_election.cc:302] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f51f5b0dc6d435f8dee647b2a20372a; no voters:
I20260928 06:44:04.790630 25315 leader_election.cc:288] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 06:44:04.790830 25319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 3 FOLLOWER]: Leader election won for term 3
I20260928 06:44:04.791078 25319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [term 3 LEADER]: Becoming Leader. State: Replica: 3f51f5b0dc6d435f8dee647b2a20372a, State: Running, Role: LEADER
I20260928 06:44:04.791437 25319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } }
I20260928 06:44:04.791932 25315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:04.792549 25319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } } }
I20260928 06:44:04.792750 25319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:04.792987 25319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f51f5b0dc6d435f8dee647b2a20372a. Latest consensus state: current_term: 3 leader_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f51f5b0dc6d435f8dee647b2a20372a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39155 } } }
I20260928 06:44:04.793123 25319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:04.793411 25328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:04.798699 25328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:04.799420 25328 catalog_manager.cc:1259] Loaded cluster ID: 8b5a88b2ef2b447985f35f88716c6942
I20260928 06:44:04.799532 25328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:04.802127 25328 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:04.802948 25328 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Loaded TSK: 0
I20260928 06:44:04.803093 25328 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f51f5b0dc6d435f8dee647b2a20372a: Loaded TSK: 1
F20260928 06:44:04.804276 25328 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f46b66789ad google::LogMessage::Fail() at ??:0
@ 0x7f46b667b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f46b66784de google::LogMessage::Flush() at ??:0
@ 0x7f46b6679a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f46b933c18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f46b9334f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f46c0fdd7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f46c0fb799c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f46c0fb8178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f46c0ff9df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f46c101882b std::function<>::operator()() at ??:0
@ 0x7f46c0fb7b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f46c0fb8f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f46c0fb5579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f46c0ff9640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f46b88ca3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f46b88cac83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f46b88cc3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f46b88bb512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f46b9eff6db start_thread at ??:0
@ 0x7f46b517661f clone at ??:0
W20260928 06:44:04.906845 25317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:04.907235 25317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:04.907317 25317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:04.913214 25317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:04.913282 25317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:04.913369 25317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:44:04.921235 25317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:33611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=39103
--tserver_master_addrs=127.8.151.254:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:04.922570 25317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:04.923660 25317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:04.934491 25362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:04.934505 25363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:04.935074 25317 server_base.cc:1034] running on GCE node
W20260928 06:44:04.935111 25365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:04.935527 25317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:04.936156 25317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:04.937325 25317 hybrid_clock.cc:648] HybridClock initialized: now 1790577844937292 us; error 50 us; skew 500 ppm
Sep 28 06:44:04 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577844, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:04.940866 25317 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:44:04.941957 25317 webserver.cc:460] Webserver started at http://127.8.151.193:39103/ using document root <none> and password file <none>
I20260928 06:44:04.942687 25317 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:04.942785 25317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:04.948042 25317 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260928 06:44:04.950628 25372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:04.952201 25317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 06:44:04.952318 25317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "18ca0400c67b46e3aa45e0e4f355cc0f"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "e1fa75c648384d39a20a390f09a7fda1"
server_key_iv: "decd9a2064effe14422f08b3ea286632"
server_key_version: "encryptionkey@0"
I20260928 06:44:04.953006 25317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:04.968616 25317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:04.972576 25317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:04.972782 25317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:04.973472 25317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:04.974821 25379 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 06:44:04.978948 25317 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 06:44:04.979022 25317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260928 06:44:04.979084 25317 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 06:44:04.981313 25317 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 06:44:04.981372 25317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260928 06:44:04.981673 25379 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Bootstrap starting.
I20260928 06:44:04.991906 25317 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:33611
I20260928 06:44:04.991954 25486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:33611 every 8 connection(s)
I20260928 06:44:04.993053 25317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:44:04.999944 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 25317
I20260928 06:44:05.000434 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:45037
--local_ip_for_outbound_sockets=127.8.151.194
--tserver_master_addrs=127.8.151.254:39155
--webserver_port=39433
--webserver_interface=127.8.151.194
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:44:05.003258 25379 log.cc:824] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:05.012490 25379 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:44:05.012928 25379 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Bootstrap complete.
I20260928 06:44:05.014367 25379 ts_tablet_manager.cc:1397] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Time spent bootstrapping tablet: real 0.033s user 0.022s sys 0.008s
I20260928 06:44:05.017758 25379 raft_consensus.cc:348] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:44:05.018307 25379 raft_consensus.cc:729] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18ca0400c67b46e3aa45e0e4f355cc0f, State: Initialized, Role: FOLLOWER
I20260928 06:44:05.018842 25379 consensus_queue.cc:260] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:44:05.020025 25379 ts_tablet_manager.cc:1428] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
W20260928 06:44:05.048625 25489 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:04.995261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:36341)
0928 06:44:04.995788 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:04.995858 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:04.996944 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:05.047862 (+ 50918us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":141,"threads_started":1}
W20260928 06:44:05.049216 25487 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:39155 (0 consecutive failures): Network error: Failed to ping master at 127.8.151.254:39155: Client connection negotiation failed: client connection to 127.8.151.254:39155: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 06:44:05.049997 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:05.049730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:36541)
0928 06:44:05.049868 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:05.049935 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 06:44:05.050628 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:05.050442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:34473)
0928 06:44:05.050538 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:05.050581 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 06:44:05.050781 25487 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 06:44:05.122974 25490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:05.123284 25490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:05.123332 25490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:05.127491 25490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:05.127549 25490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:05.127611 25490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:44:05.133066 25490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:45037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=39433
--tserver_master_addrs=127.8.151.254:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:05.134125 25490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:05.135129 25490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:05.145025 25500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:05.144972 25497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:05.144972 25498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:05.145236 25490 server_base.cc:1034] running on GCE node
I20260928 06:44:05.145725 25490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:05.146529 25490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:05.147744 25490 hybrid_clock.cc:648] HybridClock initialized: now 1790577845147713 us; error 46 us; skew 500 ppm
Sep 28 06:44:05 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577845, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:05.151006 25490 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:44:05.151925 25490 webserver.cc:460] Webserver started at http://127.8.151.194:39433/ using document root <none> and password file <none>
I20260928 06:44:05.152518 25490 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:05.152580 25490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:05.156476 25490 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 06:44:05.158809 25507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:05.160240 25490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:05.160396 25490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "54aab4f020e6407e8a143ce6eb56d9bc"
format_stamp: "Formatted at 2026-09-28 06:43:22 on dist-test-slave-gtr8"
server_key: "549089452f305a42073c7ebfd3c45fd7"
server_key_iv: "82e94efe59702be7324d588581a4c883"
server_key_version: "encryptionkey@0"
I20260928 06:44:05.160944 25490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:05.206063 25490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:05.209335 25490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:05.209508 25490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:05.210111 25490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:05.211231 25514 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 06:44:05.216574 25490 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 06:44:05.216650 25490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260928 06:44:05.216717 25490 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 06:44:05.218374 25490 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 06:44:05.218420 25490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260928 06:44:05.218724 25514 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Bootstrap starting.
I20260928 06:44:05.225772 25514 log.cc:824] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:05.228500 25490 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:45037
I20260928 06:44:05.228533 25622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:45037 every 8 connection(s)
I20260928 06:44:05.229393 25490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:44:05.232702 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:05.231360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:43233)
0928 06:44:05.231851 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:05.231981 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":119,"threads_started":1}
W20260928 06:44:05.233103 25623 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:39155 (0 consecutive failures): Network error: Failed to ping master at 127.8.151.254:39155: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
I20260928 06:44:05.233736 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:05.233535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:47149)
0928 06:44:05.233636 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:05.233684 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260928 06:44:05.234284 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:05.234107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:33253)
0928 06:44:05.234194 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:05.234237 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 06:44:05.234427 25623 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 06:44:05.236428 25514 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:44:05.236852 25514 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Bootstrap complete.
I20260928 06:44:05.237205 25514 ts_tablet_manager.cc:1397] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.003s
I20260928 06:44:05.239400 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 25490
I20260928 06:44:05.239805 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:44671
--local_ip_for_outbound_sockets=127.8.151.195
--tserver_master_addrs=127.8.151.254:39155
--webserver_port=39525
--webserver_interface=127.8.151.195
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:44:05.240729 25514 raft_consensus.cc:348] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:44:05.241273 25514 raft_consensus.cc:729] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54aab4f020e6407e8a143ce6eb56d9bc, State: Initialized, Role: FOLLOWER
I20260928 06:44:05.241817 25514 consensus_queue.cc:260] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:44:05.242895 25514 ts_tablet_manager.cc:1428] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260928 06:44:05.243088 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:05.242766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:60045)
0928 06:44:05.242949 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:05.243005 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260928 06:44:05.243875 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:05.243607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:57479)
0928 06:44:05.243735 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:05.243811 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260928 06:44:05.350483 25626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:05.350742 25626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:05.350785 25626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:05.354758 25626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:05.354815 25626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:05.354880 25626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:44:05.360250 25626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:44671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=39525
--tserver_master_addrs=127.8.151.254:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:05.361336 25626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:05.362375 25626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:05.371888 25632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:05.372047 25626 server_base.cc:1034] running on GCE node
W20260928 06:44:05.371932 25635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:05.371888 25633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:05.372577 25626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:05.373344 25626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:05.374559 25626 hybrid_clock.cc:648] HybridClock initialized: now 1790577845374530 us; error 41 us; skew 500 ppm
Sep 28 06:44:05 dist-test-slave-gtr8 krb5kdc[24172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577845, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:05.377712 25626 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:44:05.378717 25626 webserver.cc:460] Webserver started at http://127.8.151.195:39525/ using document root <none> and password file <none>
I20260928 06:44:05.379392 25626 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:05.379477 25626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:05.384121 25626 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:44:05.386447 25642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:05.387864 25626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 06:44:05.387980 25626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "e799d9ba35154838b05533257e3de6cc"
format_stamp: "Formatted at 2026-09-28 06:43:23 on dist-test-slave-gtr8"
server_key: "4d6869c7f7ff7842de004974f45e9556"
server_key_iv: "96a437a92370187a2ef4fe4e6aaf8a60"
server_key_version: "encryptionkey@0"
I20260928 06:44:05.388639 25626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:05.403220 25626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:05.406596 25626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:05.406773 25626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:05.407511 25626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:05.408454 25626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:05.408514 25626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:05.408571 25626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:05.408595 25626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:05.418990 25626 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:44671
I20260928 06:44:05.419024 25755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:44671 every 8 connection(s)
I20260928 06:44:05.420246 25626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:44:05.424208 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:05.422597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:51323)
0928 06:44:05.423371 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:05.423482 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":127,"threads_started":1}
W20260928 06:44:05.424628 25756 heartbeater.cc:644] Failed to heartbeat to 127.8.151.254:39155 (0 consecutive failures): Network error: Failed to ping master at 127.8.151.254:39155: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
I20260928 06:44:05.425230 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:05.425038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:44213)
0928 06:44:05.425131 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:05.425177 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 06:44:05.425843 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:05.425644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:55559)
0928 06:44:05.425734 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:05.425779 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 06:44:05.425982 25756 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 06:44:05.426700 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 25626
I20260928 06:44:06.051779 25760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.051358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:55693)
0928 06:44:06.051586 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:06.051647 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260928 06:44:06.228225 25761 raft_consensus.cc:482] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:44:06.228446 25761 raft_consensus.cc:504] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:44:06.229498 25761 leader_election.cc:288] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671), 54aab4f020e6407e8a143ce6eb56d9bc (127.8.151.194:45037)
Sep 28 06:44:06 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577844, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.194@KRBTEST.COM
Sep 28 06:44:06 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577844, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.195@KRBTEST.COM
I20260928 06:44:06.245149 25765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.244549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:45003)
0928 06:44:06.244951 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:06.245018 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":101,"threads_started":1}
I20260928 06:44:06.245756 25760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.229676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:44671 (local address 127.8.151.193:55251)
0928 06:44:06.229873 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:06.229887 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:06.229992 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:06.232462 (+ 2470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:06.232468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:06.233310 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:06.233588 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:06.233608 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:06.235493 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:06.235496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:06.236916 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:06.236931 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:06.237867 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:06.237871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:06.238058 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:06.238687 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:06.238706 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:06.240881 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:06.243189 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:06.243192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:06.243195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:06.243484 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:06.243843 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:06.243847 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:06.243849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:06.243988 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:06.244780 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:06.244784 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:06.245070 (+ 286us) client_negotiation.cc:769] Sending connection context
0928 06:44:06.245272 (+ 202us) client_negotiation.cc:240] Negotiation successful
0928 06:44:06.245571 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":116,"spinlock_wait_cycles":24832}
I20260928 06:44:06.246133 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.229852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:45037 (local address 127.8.151.193:42031)
0928 06:44:06.230364 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:06.230377 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:06.230468 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:06.232462 (+ 1994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:06.232468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:06.232968 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:06.233588 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:06.233608 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:06.235699 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:06.235702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:06.236888 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:06.236899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:06.237852 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:06.237855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:06.238058 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:06.238726 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:06.238738 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:06.240511 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:06.243004 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:06.243010 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:06.243025 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:06.243496 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:06.243906 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:06.243910 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:06.243912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:06.244000 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:06.244769 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:06.244776 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:06.245715 (+ 939us) client_negotiation.cc:769] Sending connection context
0928 06:44:06.245820 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 06:44:06.245967 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":875,"spinlock_wait_cycles":11520,"thread_start_us":74,"threads_started":1}
I20260928 06:44:06.246765 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.230623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:42031 (local address 127.8.151.194:45037)
0928 06:44:06.230943 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:06.230958 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:06.231000 (+ 42us) server_negotiation.cc:407] Connection header received
0928 06:44:06.231295 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:06.231301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:06.231751 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:06.233811 (+ 2060us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:06.234193 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:06.235563 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:06.237028 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:06.237720 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:06.237881 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:06.240702 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:06.240708 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:06.240724 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:06.240768 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:06.242870 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:06.243629 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:06.243634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:06.243640 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:06.243794 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:06.244102 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:06.244105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:06.244107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:06.244634 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:06.244867 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:06.246268 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0928 06:44:06.246588 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":58,"threads_started":1}
I20260928 06:44:06.246899 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.230284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:55251 (local address 127.8.151.195:44671)
0928 06:44:06.230943 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:06.230958 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:06.231000 (+ 42us) server_negotiation.cc:407] Connection header received
0928 06:44:06.231843 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:06.231851 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:06.232221 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:06.232720 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:06.233782 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:06.235341 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:06.237032 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:06.237720 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:06.238383 (+ 663us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:06.241093 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:06.241100 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:06.241113 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:06.241145 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:06.243058 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:06.243602 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:06.243608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:06.243614 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:06.243745 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:06.244082 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:06.244086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:06.244088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:06.244644 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:06.246341 (+ 1697us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:06.246480 (+ 139us) server_negotiation.cc:299] Negotiation successful
0928 06:44:06.246728 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":77,"threads_started":1}
W20260928 06:44:06.249365 25376 leader_election.cc:341] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671): Not found: Tablet not found: 33704e80a9bd4f0090c03694cafb2629
I20260928 06:44:06.249922 25576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33704e80a9bd4f0090c03694cafb2629" candidate_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" is_pre_election: true
I20260928 06:44:06.250216 25576 raft_consensus.cc:2454] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18ca0400c67b46e3aa45e0e4f355cc0f in term 1.
I20260928 06:44:06.250964 25375 leader_election.cc:302] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18ca0400c67b46e3aa45e0e4f355cc0f, 54aab4f020e6407e8a143ce6eb56d9bc; no voters: e799d9ba35154838b05533257e3de6cc
I20260928 06:44:06.251274 25761 raft_consensus.cc:2781] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 06:44:06.251362 25761 raft_consensus.cc:482] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:44:06.251410 25761 raft_consensus.cc:3037] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 1 FOLLOWER]: Advancing to term 2
I20260928 06:44:06.253666 25761 raft_consensus.cc:504] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:44:06.254033 25761 leader_election.cc:288] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [CANDIDATE]: Term 2 election: Requested vote from peers e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671), 54aab4f020e6407e8a143ce6eb56d9bc (127.8.151.194:45037)
I20260928 06:44:06.254400 25576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33704e80a9bd4f0090c03694cafb2629" candidate_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "54aab4f020e6407e8a143ce6eb56d9bc"
I20260928 06:44:06.254516 25576 raft_consensus.cc:3037] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 1 FOLLOWER]: Advancing to term 2
W20260928 06:44:06.254707 25376 leader_election.cc:341] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671): Not found: Tablet not found: 33704e80a9bd4f0090c03694cafb2629
I20260928 06:44:06.256655 25576 raft_consensus.cc:2454] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18ca0400c67b46e3aa45e0e4f355cc0f in term 2.
I20260928 06:44:06.256978 25375 leader_election.cc:302] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18ca0400c67b46e3aa45e0e4f355cc0f, 54aab4f020e6407e8a143ce6eb56d9bc; no voters: e799d9ba35154838b05533257e3de6cc
I20260928 06:44:06.257207 25761 raft_consensus.cc:2781] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 2 FOLLOWER]: Leader election won for term 2
I20260928 06:44:06.257426 25761 raft_consensus.cc:686] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [term 2 LEADER]: Becoming Leader. State: Replica: 18ca0400c67b46e3aa45e0e4f355cc0f, State: Running, Role: LEADER
I20260928 06:44:06.257786 25761 consensus_queue.cc:237] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:44:06.258198 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.257939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:60763)
0928 06:44:06.258064 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:06.258124 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 06:44:06.426877 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.426432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:60253)
0928 06:44:06.426700 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:06.426762 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":71,"threads_started":1}
I20260928 06:44:06.798578 25576 raft_consensus.cc:1261] T 33704e80a9bd4f0090c03694cafb2629 P 54aab4f020e6407e8a143ce6eb56d9bc [term 2 FOLLOWER]: Refusing update from remote peer 18ca0400c67b46e3aa45e0e4f355cc0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260928 06:44:06.799293 25769 consensus_queue.cc:1035] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260928 06:44:06.801389 25771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.801025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:45985)
0928 06:44:06.801217 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:06.801279 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":60,"threads_started":1}
W20260928 06:44:06.802065 25376 consensus_peers.cc:487] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f -> Peer e799d9ba35154838b05533257e3de6cc (127.8.151.195:44671): Couldn't send request to peer e799d9ba35154838b05533257e3de6cc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33704e80a9bd4f0090c03694cafb2629. This is attempt 1: this message will repeat every 5th retry.
I20260928 06:44:06.803836 25776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.803490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:51565)
0928 06:44:06.803681 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:06.803743 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
I20260928 06:44:06.804283 25771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:06.804017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:34469)
0928 06:44:06.804172 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:06.804221 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260928 06:44:07.417209 25778 ts_tablet_manager.cc:927] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Initiating tablet copy from peer 18ca0400c67b46e3aa45e0e4f355cc0f (127.8.151.193:33611)
I20260928 06:44:07.417795 25778 tablet_copy_client.cc:316] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: tablet copy: Beginning tablet copy session from remote peer at address 127.8.151.193:33611
Sep 28 06:44:07 dist-test-slave-gtr8 krb5kdc[24172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577845, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.193@KRBTEST.COM
I20260928 06:44:07.428074 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:07.427593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:52725)
0928 06:44:07.427910 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:07.427967 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":119,"threads_started":1}
I20260928 06:44:07.429374 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:07.418334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:33611 (local address 127.8.151.195:58553)
0928 06:44:07.418622 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:07.418647 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:07.418790 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:07.419270 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:07.419277 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:07.419795 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:07.419948 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:07.419955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:07.421299 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:07.421303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:07.422320 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:07.422331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:07.423099 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:07.423102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:07.423244 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:07.423849 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:07.423861 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:07.425453 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:07.427566 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:07.427576 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:07.427582 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:07.427873 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:07.428157 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:07.428160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:07.428162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:07.428255 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:07.428837 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:07.428846 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:07.428952 (+ 106us) client_negotiation.cc:769] Sending connection context
0928 06:44:07.429070 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 06:44:07.429197 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":96,"threads_started":1}
I20260928 06:44:07.429461 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:07.418450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:58553 (local address 127.8.151.193:33611)
0928 06:44:07.418738 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:07.418746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:07.418772 (+ 26us) server_negotiation.cc:407] Connection header received
0928 06:44:07.418874 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:07.418880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:07.419100 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:07.419260 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:07.420091 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:07.421176 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:07.422444 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:07.422982 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:07.423081 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:07.425591 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:07.425601 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:07.425615 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:07.425641 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:07.427453 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:07.427980 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:07.427987 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:07.427988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:07.428071 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:07.428402 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:07.428406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:07.428407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:07.428706 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:07.428847 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:07.429169 (+ 322us) server_negotiation.cc:299] Negotiation successful
0928 06:44:07.429315 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260928 06:44:07.430732 25443 tablet_copy_service.cc:140] P 18ca0400c67b46e3aa45e0e4f355cc0f: Received BeginTabletCopySession request for tablet 33704e80a9bd4f0090c03694cafb2629 from peer e799d9ba35154838b05533257e3de6cc ({username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:58553)
I20260928 06:44:07.430910 25443 tablet_copy_service.cc:161] P 18ca0400c67b46e3aa45e0e4f355cc0f: Beginning new tablet copy session on tablet 33704e80a9bd4f0090c03694cafb2629 from peer e799d9ba35154838b05533257e3de6cc at {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:58553: session id = e799d9ba35154838b05533257e3de6cc-33704e80a9bd4f0090c03694cafb2629
I20260928 06:44:07.432083 25443 tablet_copy_source_session.cc:205] T 33704e80a9bd4f0090c03694cafb2629 P 18ca0400c67b46e3aa45e0e4f355cc0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 06:44:07.435957 25778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33704e80a9bd4f0090c03694cafb2629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:07.442276 25778 tablet_copy_client.cc:791] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: tablet copy: Starting download of 0 data blocks...
I20260928 06:44:07.442474 25778 tablet_copy_client.cc:655] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: tablet copy: Starting download of 1 WAL segments...
I20260928 06:44:07.444298 25778 tablet_copy_client.cc:531] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 06:44:07.446058 25778 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Bootstrap starting.
I20260928 06:44:07.451576 25778 log.cc:824] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:07.458344 25778 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 06:44:07.458626 25778 tablet_bootstrap.cc:492] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Bootstrap complete.
I20260928 06:44:07.458948 25778 ts_tablet_manager.cc:1397] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260928 06:44:07.461268 25778 raft_consensus.cc:348] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:44:07.461522 25778 raft_consensus.cc:729] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e799d9ba35154838b05533257e3de6cc, State: Initialized, Role: FOLLOWER
I20260928 06:44:07.461975 25778 consensus_queue.cc:260] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e799d9ba35154838b05533257e3de6cc" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44671 } } peers { permanent_uuid: "54aab4f020e6407e8a143ce6eb56d9bc" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 45037 } } peers { permanent_uuid: "18ca0400c67b46e3aa45e0e4f355cc0f" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 33611 } }
I20260928 06:44:07.462944 25778 ts_tablet_manager.cc:1428] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20260928 06:44:07.463323 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:07.462985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:56883)
0928 06:44:07.463184 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:07.463251 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260928 06:44:07.463668 25443 tablet_copy_service.cc:342] P 18ca0400c67b46e3aa45e0e4f355cc0f: Request end of tablet copy session e799d9ba35154838b05533257e3de6cc-33704e80a9bd4f0090c03694cafb2629 received from {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:58553
I20260928 06:44:07.463778 25443 tablet_copy_service.cc:433] P 18ca0400c67b46e3aa45e0e4f355cc0f: ending tablet copy session e799d9ba35154838b05533257e3de6cc-33704e80a9bd4f0090c03694cafb2629 on tablet 33704e80a9bd4f0090c03694cafb2629 with peer e799d9ba35154838b05533257e3de6cc
I20260928 06:44:07.464046 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:07.463774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:38415)
0928 06:44:07.463917 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:07.463988 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 06:44:07.805015 25785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:07.804596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:56573)
0928 06:44:07.804851 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:07.804913 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":74,"threads_started":1}
I20260928 06:44:07.805439 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:07.804969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:60203)
0928 06:44:07.805277 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:07.805343 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":68,"threads_started":1}
I20260928 06:44:07.919863 25785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:07.919595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:37999)
0928 06:44:07.919727 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:07.919781 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 06:44:07.920156 25710 raft_consensus.cc:1203] T 33704e80a9bd4f0090c03694cafb2629 P e799d9ba35154838b05533257e3de6cc [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260928 06:44:07.921100 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:07.920814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:35297)
0928 06:44:07.920979 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:07.921026 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260928 06:44:08.806684 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:08.806203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:56427)
0928 06:44:08.806522 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:08.806577 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":91,"threads_started":1}
I20260928 06:44:08.921229 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:08.920699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:55831)
0928 06:44:08.921057 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:08.921116 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":87,"threads_started":1}
I20260928 06:44:08.922267 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:08.921784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:58039)
0928 06:44:08.922102 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:08.922155 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":74,"threads_started":1}
I20260928 06:44:09.807953 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:09.807516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:44099)
0928 06:44:09.807783 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:09.807843 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":77,"threads_started":1}
I20260928 06:44:09.922385 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:09.921995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:43879)
0928 06:44:09.922240 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:09.922294 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":85,"threads_started":1}
I20260928 06:44:09.923311 25794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:09.922887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:47691)
0928 06:44:09.923128 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:09.923215 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":60,"threads_started":1}
I20260928 06:44:10.809279 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:10.808864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:52729)
0928 06:44:10.809119 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:10.809180 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
I20260928 06:44:10.923676 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:10.923253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:35461)
0928 06:44:10.923491 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:10.923579 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260928 06:44:10.924574 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:10.924185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:39567)
0928 06:44:10.924414 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:10.924473 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":79,"threads_started":1}
I20260928 06:44:10.924587 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:10.924292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:39791)
0928 06:44:10.924465 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:10.924510 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260928 06:44:11.810442 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:11.810038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:59893)
0928 06:44:11.810283 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:11.810343 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":84,"threads_started":1}
I20260928 06:44:11.925820 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:11.925353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:60047)
0928 06:44:11.925611 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:11.925669 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
I20260928 06:44:11.925873 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:11.925365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:42561)
0928 06:44:11.925674 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:11.925754 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":72,"threads_started":1}
I20260928 06:44:12.811648 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:12.811217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:46177)
0928 06:44:12.811492 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:12.811551 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":90,"threads_started":1}
I20260928 06:44:12.926973 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:12.926545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:49325)
0928 06:44:12.926799 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:12.926860 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":85,"threads_started":1}
I20260928 06:44:12.927034 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:12.926595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:37297)
0928 06:44:12.926900 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:12.926955 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":92,"threads_started":1}
I20260928 06:44:13.813097 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:13.812570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:39963)
0928 06:44:13.812879 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:13.812963 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":93,"threads_started":1}
I20260928 06:44:13.928227 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:13.927763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:41451)
0928 06:44:13.928043 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:13.928112 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":87,"threads_started":1}
I20260928 06:44:13.928642 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:13.927763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:46637)
0928 06:44:13.928488 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:13.928545 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":88,"threads_started":1}
I20260928 06:44:14.814523 25808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:14.814018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:60443)
0928 06:44:14.814325 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:14.814421 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":94,"threads_started":1}
I20260928 06:44:14.929384 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:14.928899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:60899)
0928 06:44:14.929194 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:14.929257 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":99,"threads_started":1}
I20260928 06:44:14.929780 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:14.929251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:35069)
0928 06:44:14.929617 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:14.929685 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
I20260928 06:44:15.815790 25812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:15.815368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:47655)
0928 06:44:15.815629 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:15.815693 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260928 06:44:15.930573 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:15.930107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:56951)
0928 06:44:15.930343 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:15.930459 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":74,"threads_started":1}
I20260928 06:44:15.930943 25814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:15.930513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:35737)
0928 06:44:15.930771 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:15.930841 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":87,"threads_started":1}
I20260928 06:44:16.817049 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:16.816618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:40037)
0928 06:44:16.816886 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:16.816947 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":86,"threads_started":1}
I20260928 06:44:16.931789 25816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:16.931392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:45139)
0928 06:44:16.931640 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:16.931697 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":79,"threads_started":1}
I20260928 06:44:16.932210 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:16.931711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:54145)
0928 06:44:16.932049 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:16.932111 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":95,"threads_started":1}
I20260928 06:44:17.818310 25818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:17.817864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:34367)
0928 06:44:17.818154 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:17.818215 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":124,"threads_started":1}
I20260928 06:44:17.932965 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:17.932569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:52567)
0928 06:44:17.932817 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:17.932870 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":83,"threads_started":1}
I20260928 06:44:17.933410 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:17.932934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:51157)
0928 06:44:17.933285 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:17.933334 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":113,"threads_started":1}
I20260928 06:44:18.819682 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:18.819217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:55779)
0928 06:44:18.819505 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:18.819577 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":88,"threads_started":1}
I20260928 06:44:18.934367 25822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:18.933891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:48091)
0928 06:44:18.934184 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:18.934252 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":96,"threads_started":1}
I20260928 06:44:18.934718 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:18.934030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:40367)
0928 06:44:18.934298 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:18.934360 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":69,"threads_started":1}
I20260928 06:44:19.820919 25824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:19.820489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.194:51627)
0928 06:44:19.820757 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:19.820818 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":92,"threads_started":1}
I20260928 06:44:19.935544 25825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:19.935102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.195:37595)
0928 06:44:19.935389 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:19.935447 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":103,"threads_started":1}
I20260928 06:44:19.936008 25826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:19.935520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39155 (local address 127.8.151.193:53257)
0928 06:44:19.935853 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:19.935912 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.254:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":58,"threads_started":1}
I20260928 06:44:20.434826 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 25317
I20260928 06:44:20.440183 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 25490
I20260928 06:44:20.445206 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 25626
2026-09-28T06:44:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59444 ms)
[----------] 30 tests from SecurityITest (254470 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:20 dist-test-slave-gtr8 krb5kdc[25830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:20 dist-test-slave-gtr8 krb5kdc[25830](info): set up 2 sockets
Sep 28 06:44:20 dist-test-slave-gtr8 krb5kdc[25830](info): commencing operation
krb5kdc: starting...
W20260928 06:44:21.419507 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:21 dist-test-slave-gtr8 krb5kdc[25830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:21.546747 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:38957
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:38957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:21.700495 25843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:21.700958 25843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:21.701066 25843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:21.707077 25843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:21.707185 25843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:21.707218 25843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:21.707244 25843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:21.707278 25843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:21.714525 25843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:38957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:38957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:21.715919 25843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:21.717252 25843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:44:21.726612 25843 server_base.cc:1034] running on GCE node
W20260928 06:44:21.726605 25848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:21.726708 25849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:21.726871 25851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:21.727494 25843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:21.728555 25843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:21.729730 25843 hybrid_clock.cc:648] HybridClock initialized: now 1790577861729702 us; error 43 us; skew 500 ppm
Sep 28 06:44:21 dist-test-slave-gtr8 krb5kdc[25830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577861, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:21.732566 25843 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:21.733457 25843 webserver.cc:460] Webserver started at http://127.8.151.254:36789/ using document root <none> and password file <none>
I20260928 06:44:21.733980 25843 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:21.734037 25843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:21.734232 25843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:21.736281 25843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "70f410d1c27c4970b3ce08a17ccb5c2b"
format_stamp: "Formatted at 2026-09-28 06:44:21 on dist-test-slave-gtr8"
server_key: "4636c13a0f8eb3a5903be95ea53038d9"
server_key_iv: "59551a823e9f7ba14c64d4ed8e5f34ef"
server_key_version: "encryptionkey@0"
I20260928 06:44:21.736810 25843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "70f410d1c27c4970b3ce08a17ccb5c2b"
format_stamp: "Formatted at 2026-09-28 06:44:21 on dist-test-slave-gtr8"
server_key: "4636c13a0f8eb3a5903be95ea53038d9"
server_key_iv: "59551a823e9f7ba14c64d4ed8e5f34ef"
server_key_version: "encryptionkey@0"
I20260928 06:44:21.741257 25843 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:44:21.743638 25858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:21.744937 25843 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:44:21.745046 25843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "70f410d1c27c4970b3ce08a17ccb5c2b"
format_stamp: "Formatted at 2026-09-28 06:44:21 on dist-test-slave-gtr8"
server_key: "4636c13a0f8eb3a5903be95ea53038d9"
server_key_iv: "59551a823e9f7ba14c64d4ed8e5f34ef"
server_key_version: "encryptionkey@0"
I20260928 06:44:21.745138 25843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:21.766431 25843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:21.769557 25843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:21.769731 25843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:21.776584 25910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:38957 every 8 connection(s)
I20260928 06:44:21.776585 25843 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:38957
I20260928 06:44:21.777757 25843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:21.780606 25911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:21.783680 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 25843
I20260928 06:44:21.783797 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:21.784049 8799 external_mini_cluster.cc:1428] Setting key 6c1ceb1025a4998fba11c3748f1a12f3
I20260928 06:44:21.786304 25911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b: Bootstrap starting.
I20260928 06:44:21.788902 25911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:21.789752 25911 log.cc:824] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:21.791613 25911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b: No bootstrap required, opened a new log
Sep 28 06:44:21 dist-test-slave-gtr8 krb5kdc[25830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:21.795166 25911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f410d1c27c4970b3ce08a17ccb5c2b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38957 } }
I20260928 06:44:21.795440 25911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:21.795487 25911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70f410d1c27c4970b3ce08a17ccb5c2b, State: Initialized, Role: FOLLOWER
I20260928 06:44:21.796041 25911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f410d1c27c4970b3ce08a17ccb5c2b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38957 } }
I20260928 06:44:21.796197 25911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:21.796277 25911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:21.796375 25911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:21.797386 25911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f410d1c27c4970b3ce08a17ccb5c2b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38957 } }
I20260928 06:44:21.797761 25911 leader_election.cc:302] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70f410d1c27c4970b3ce08a17ccb5c2b; no voters:
I20260928 06:44:21.798120 25911 leader_election.cc:288] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:21.798197 25916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:21.798398 25916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [term 1 LEADER]: Becoming Leader. State: Replica: 70f410d1c27c4970b3ce08a17ccb5c2b, State: Running, Role: LEADER
I20260928 06:44:21.798750 25916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f410d1c27c4970b3ce08a17ccb5c2b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38957 } }
I20260928 06:44:21.799316 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:21.785402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52396 (local address 127.8.151.254:38957)
0928 06:44:21.785810 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:21.785828 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:21.785870 (+ 42us) server_negotiation.cc:407] Connection header received
0928 06:44:21.786650 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:21.786672 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:21.787069 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:21.787459 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:21.788063 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:21.789559 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:21.790448 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:21.790943 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:21.791088 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:21.793581 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:21.793587 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:21.793602 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:21.793633 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:21.796007 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:21.796574 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:21.796579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:21.796585 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:21.796718 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:21.796958 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:21.796961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:21.796962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:21.797481 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:21.797657 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:21.797874 (+ 217us) server_negotiation.cc:299] Negotiation successful
0928 06:44:21.798336 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":114,"threads_started":1}
I20260928 06:44:21.799397 25911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:21.800426 25917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70f410d1c27c4970b3ce08a17ccb5c2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f410d1c27c4970b3ce08a17ccb5c2b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38957 } } }
I20260928 06:44:21.800544 25917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:21.800949 25918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70f410d1c27c4970b3ce08a17ccb5c2b. Latest consensus state: current_term: 1 leader_uuid: "70f410d1c27c4970b3ce08a17ccb5c2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f410d1c27c4970b3ce08a17ccb5c2b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 38957 } } }
I20260928 06:44:21.801806 25918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:21.804265 25931 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:21.804355 25931 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:21.804495 25925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:21.805338 25925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:21.811229 25925 catalog_manager.cc:1347] Generated new cluster ID: afc29ff8f99b4c8084b77f8d05766fce
I20260928 06:44:21.811288 25925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:21.826810 25925 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:21.828116 25925 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:21.841071 25925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b: Generated new TSK 0
I20260928 06:44:21.841871 25925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:44:21.874231 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38957
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:21.983655 25937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:21.983924 25937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:21.983989 25937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:21.988092 25937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:21.988154 25937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:21.988229 25937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:44:21.993427 25937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:21.994417 25937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:21.995445 25937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:22.004972 25945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:22.004976 25942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:22.005108 25937 server_base.cc:1034] running on GCE node
W20260928 06:44:22.005255 25943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:22.005671 25937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:22.006283 25937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:22.007436 25937 hybrid_clock.cc:648] HybridClock initialized: now 1790577862007401 us; error 54 us; skew 500 ppm
Sep 28 06:44:22 dist-test-slave-gtr8 krb5kdc[25830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577862, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:22.010012 25937 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:44:22.010849 25937 webserver.cc:460] Webserver started at http://127.8.151.193:36557/ using document root <none> and password file <none>
I20260928 06:44:22.011410 25937 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:22.011472 25937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:22.011658 25937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:22.013659 25937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "c1c8f734815747a0b86fce2a60321f9e"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "2130034b3b5d7d93d9d5eaee4ec7503d"
server_key_iv: "07d65154ce55a0950e02a5c0879aa19f"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.014195 25937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "c1c8f734815747a0b86fce2a60321f9e"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "2130034b3b5d7d93d9d5eaee4ec7503d"
server_key_iv: "07d65154ce55a0950e02a5c0879aa19f"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.017877 25937 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260928 06:44:22.020399 25952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:22.021807 25937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:44:22.021919 25937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "c1c8f734815747a0b86fce2a60321f9e"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "2130034b3b5d7d93d9d5eaee4ec7503d"
server_key_iv: "07d65154ce55a0950e02a5c0879aa19f"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.022027 25937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:22.041672 25937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:22.044592 25937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:22.044755 25937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:22.045353 25937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:22.046160 25937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:22.046222 25937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:22.046262 25937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:22.046289 25937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:22.055527 25937 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:38055
I20260928 06:44:22.055588 26065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:38055 every 8 connection(s)
I20260928 06:44:22.056447 25937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:44:22.060410 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 25937
I20260928 06:44:22.060535 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:44:22.060823 8799 external_mini_cluster.cc:1428] Setting key 0b1a2961117757b9f3ffc0c464ed7a17
Sep 28 06:44:22 dist-test-slave-gtr8 krb5kdc[25830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577862, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:44:22.072220 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.058515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:34659 (local address 127.8.151.254:38957)
0928 06:44:22.058670 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.058675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.059610 (+ 935us) server_negotiation.cc:407] Connection header received
0928 06:44:22.060728 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.060731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.060786 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.060862 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:22.062345 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.062869 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.064093 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.064462 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.064537 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.067758 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:22.067763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:22.067766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:22.067796 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:22.069560 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:22.070236 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:22.070240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:22.070241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:22.070335 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:22.070771 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:22.070775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:22.070776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:22.071022 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:22.071107 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.071913 (+ 806us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.072045 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 06:44:22.073179 26068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.058893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38957 (local address 127.8.151.193:34659)
0928 06:44:22.059430 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:22.059492 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:22.060491 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:22.061054 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:22.061061 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:22.061512 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:22.062175 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.062197 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.062995 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.062997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.063947 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.063959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.064743 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.064747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.064948 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.065625 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:22.065645 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:22.067533 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:22.069708 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:22.069715 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:22.069735 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:22.070096 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:22.070449 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:22.070452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:22.070454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:22.070651 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:22.071232 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:22.071239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:22.071616 (+ 377us) client_negotiation.cc:769] Sending connection context
0928 06:44:22.071835 (+ 219us) client_negotiation.cc:240] Negotiation successful
0928 06:44:22.072205 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:44:22.074576 26066 heartbeater.cc:347] Connected to a master server at 127.8.151.254:38957
I20260928 06:44:22.074837 26066 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:22.075434 26066 heartbeater.cc:511] Master 127.8.151.254:38957 requested a full tablet report, sending...
I20260928 06:44:22.077230 25875 ts_manager.cc:194] Registered new tserver with Master: c1c8f734815747a0b86fce2a60321f9e (127.8.151.193:38055)
I20260928 06:44:22.078502 25875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:34659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:44:22.092166 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38957
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:22.200891 26071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:22.201162 26071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:22.201231 26071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:22.205696 26071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:22.205761 26071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:22.205847 26071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:44:22.211252 26071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:22.212296 26071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:22.213315 26071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:22.222756 26076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:22.222785 26077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:22.223080 26071 server_base.cc:1034] running on GCE node
W20260928 06:44:22.223176 26079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:22.223580 26071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:22.224210 26071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:22.225371 26071 hybrid_clock.cc:648] HybridClock initialized: now 1790577862225340 us; error 51 us; skew 500 ppm
Sep 28 06:44:22 dist-test-slave-gtr8 krb5kdc[25830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577862, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:22.228219 26071 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:44:22.229292 26071 webserver.cc:460] Webserver started at http://127.8.151.194:43629/ using document root <none> and password file <none>
I20260928 06:44:22.229975 26071 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:22.230067 26071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:22.230326 26071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:22.233079 26071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "683ed694922d4c9689744abcc545ad99"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "a3e3a99abdb490d1b41ece8e2426f639"
server_key_iv: "15a2b81deab02778ba774bcac146f22d"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.233767 26071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "683ed694922d4c9689744abcc545ad99"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "a3e3a99abdb490d1b41ece8e2426f639"
server_key_iv: "15a2b81deab02778ba774bcac146f22d"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.238636 26071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260928 06:44:22.241724 26086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:22.243268 26071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:22.243400 26071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "683ed694922d4c9689744abcc545ad99"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "a3e3a99abdb490d1b41ece8e2426f639"
server_key_iv: "15a2b81deab02778ba774bcac146f22d"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.243523 26071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:22.261382 26071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:22.265076 26071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:22.265292 26071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:22.266024 26071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:22.267055 26071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:22.267118 26071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:22.267211 26071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:22.267244 26071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:22.278087 26071 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:38427
I20260928 06:44:22.278100 26199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:38427 every 8 connection(s)
I20260928 06:44:22.279385 26071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
Sep 28 06:44:22 dist-test-slave-gtr8 krb5kdc[25830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577862, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:22.288830 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 26071
I20260928 06:44:22.288956 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:44:22.289276 8799 external_mini_cluster.cc:1428] Setting key 89c983b0979ebafb9e34e4a40e0cdc13
I20260928 06:44:22.294005 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.281425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:48007 (local address 127.8.151.254:38957)
0928 06:44:22.281546 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.281549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.282662 (+ 1113us) server_negotiation.cc:407] Connection header received
0928 06:44:22.283584 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.283588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.283645 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.283721 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:22.284972 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.285507 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.286626 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.286855 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.286945 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.289483 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:22.289488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:22.289490 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:22.289520 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:22.291557 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:22.292135 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:22.292138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:22.292140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:22.292237 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:22.292630 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:22.292634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:22.292636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:22.292862 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:22.292937 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.293722 (+ 785us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.293850 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 06:44:22.294987 26202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.282002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38957 (local address 127.8.151.194:48007)
0928 06:44:22.282513 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:22.282556 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:22.283382 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:22.283894 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:22.283901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:22.284251 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:22.284829 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.284845 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.285625 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.285628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.286498 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.286507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.286999 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.287002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.287190 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.287722 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:22.287738 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:22.289273 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:22.291686 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:22.291693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:22.291708 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:22.292038 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:22.292329 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:22.292332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:22.292334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:22.292511 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:22.293022 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:22.293029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:22.293368 (+ 339us) client_negotiation.cc:769] Sending connection context
0928 06:44:22.293579 (+ 211us) client_negotiation.cc:240] Negotiation successful
0928 06:44:22.293989 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":143,"threads_started":1}
I20260928 06:44:22.296460 26200 heartbeater.cc:347] Connected to a master server at 127.8.151.254:38957
I20260928 06:44:22.296721 26200 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:22.297307 26200 heartbeater.cc:511] Master 127.8.151.254:38957 requested a full tablet report, sending...
I20260928 06:44:22.298858 25875 ts_manager.cc:194] Registered new tserver with Master: 683ed694922d4c9689744abcc545ad99 (127.8.151.194:38427)
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
I20260928 06:44:22.299678 25875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:48007
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:44:22.318434 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38957
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:22.426712 26205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:22.426954 26205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:22.426994 26205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:22.431043 26205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:22.431102 26205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:22.431185 26205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:44:22.436368 26205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:38957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:22.437333 26205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:22.438323 26205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:22.447535 26210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:22.447700 26205 server_base.cc:1034] running on GCE node
W20260928 06:44:22.447546 26213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:22.447546 26211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:22.448258 26205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:22.448951 26205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:22.450165 26205 hybrid_clock.cc:648] HybridClock initialized: now 1790577862450139 us; error 39 us; skew 500 ppm
Sep 28 06:44:22 dist-test-slave-gtr8 krb5kdc[25830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577862, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:22.452975 26205 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:44:22.453959 26205 webserver.cc:460] Webserver started at http://127.8.151.195:46031/ using document root <none> and password file <none>
I20260928 06:44:22.454505 26205 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:22.454569 26205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:22.454764 26205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:22.456820 26205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "3eb34cec8cdb8281e989f410a93bfeb3"
server_key_iv: "950d7b45e435f539946eb071b8fd6276"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.457393 26205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "3eb34cec8cdb8281e989f410a93bfeb3"
server_key_iv: "950d7b45e435f539946eb071b8fd6276"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.461280 26205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260928 06:44:22.463867 26220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:22.465296 26205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:22.465408 26205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "3eb34cec8cdb8281e989f410a93bfeb3"
server_key_iv: "950d7b45e435f539946eb071b8fd6276"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.465508 26205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:22.478922 26205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:22.482260 26205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:22.482451 26205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:22.483058 26205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:22.483932 26205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:22.483979 26205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:22.484033 26205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:22.484061 26205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:22.494015 26205 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:37335
I20260928 06:44:22.494031 26333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:37335 every 8 connection(s)
I20260928 06:44:22.495036 26205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:44:22.504856 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 26205
I20260928 06:44:22.504992 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:44:22.505304 8799 external_mini_cluster.cc:1428] Setting key 149966c6a6f1a8abc3a3de3a8311d499
Sep 28 06:44:22 dist-test-slave-gtr8 krb5kdc[25830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577862, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:22.510746 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.497279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:44645 (local address 127.8.151.254:38957)
0928 06:44:22.497436 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.497440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.498418 (+ 978us) server_negotiation.cc:407] Connection header received
0928 06:44:22.500192 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.500208 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.500263 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.500368 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:22.501988 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.502525 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.504076 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.504289 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.504362 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.507249 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:22.507252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:22.507255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:22.507283 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:22.508788 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:22.509271 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:22.509274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:22.509276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:22.509357 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:22.509638 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:22.509641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:22.509642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:22.509843 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:22.509930 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.510475 (+ 545us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.510592 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 06:44:22.511749 26336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.497694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:38957 (local address 127.8.151.195:44645)
0928 06:44:22.498232 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:22.498295 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:22.499878 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:22.500562 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:22.500573 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:22.501124 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:22.501836 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.501858 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.502658 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.502661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.503964 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.503976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.504427 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.504431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.504617 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.505301 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:22.505321 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:22.507059 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:22.508901 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:22.508905 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:22.508918 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:22.509177 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:22.509426 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:22.509428 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:22.509430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:22.509558 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:22.509966 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:22.509972 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:22.510232 (+ 260us) client_negotiation.cc:769] Sending connection context
0928 06:44:22.510406 (+ 174us) client_negotiation.cc:240] Negotiation successful
0928 06:44:22.510717 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":128,"threads_started":1}
I20260928 06:44:22.512835 26334 heartbeater.cc:347] Connected to a master server at 127.8.151.254:38957
I20260928 06:44:22.513084 26334 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:22.513607 26334 heartbeater.cc:511] Master 127.8.151.254:38957 requested a full tablet report, sending...
I20260928 06:44:22.514608 25875 ts_manager.cc:194] Registered new tserver with Master: 97f1d7c34ebb4750a9c6bb5884f96dc6 (127.8.151.195:37335)
I20260928 06:44:22.515196 25875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:44645
I20260928 06:44:22.519428 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:44:22.529191 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.521556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52398 (local address 127.8.151.254:38957)
0928 06:44:22.521678 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.521682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.521851 (+ 169us) server_negotiation.cc:407] Connection header received
0928 06:44:22.521945 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.521948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.521990 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.522059 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:22.522707 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.523444 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.524343 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.524638 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.524728 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.526225 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:22.526229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:22.526231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:22.526256 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:22.527699 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:22.528078 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:22.528081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:22.528083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:22.528165 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:22.528373 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:22.528375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:22.528377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:22.528637 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:22.528755 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.528920 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.529030 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:44:22.532122 25875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:44:22.536046 25875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:44:22.552187 26348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.545493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38055 (local address 127.0.0.1:53302)
0928 06:44:22.546920 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:22.546932 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:22.547015 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:22.547685 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:22.547688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:22.547704 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:22.547787 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.547792 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.549113 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.549116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.550485 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.550505 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.551750 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.551753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.551880 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.551904 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 06:44:22.551965 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 06:44:22.552037 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1368,"mutex_wait_us":169,"thread_start_us":91,"threads_started":1}
I20260928 06:44:22.552192 26347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.545000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:38427 (local address 127.0.0.1:53840)
0928 06:44:22.545551 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:22.545566 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:22.545650 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:22.546925 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:22.546928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:22.546944 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:22.547040 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.547045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.548549 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.548552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.550357 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.550370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.551520 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.551525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.551642 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.551687 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 06:44:22.551774 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 06:44:22.551927 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":83,"thread_start_us":110,"threads_started":1}
I20260928 06:44:22.552834 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.544917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33032 (local address 127.8.151.195:37335)
0928 06:44:22.545894 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.545907 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.545931 (+ 24us) server_negotiation.cc:407] Connection header received
0928 06:44:22.546012 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.546018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.546234 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.546393 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:22.546838 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.547981 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.550331 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.551177 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.551280 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.552560 (+ 1280us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.552610 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.552699 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871,"thread_start_us":118,"threads_started":1}
I20260928 06:44:22.552435 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.544744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:37335 (local address 127.0.0.1:33032)
0928 06:44:22.545108 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:22.545166 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:22.545288 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:22.546590 (+ 1302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:22.546595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:22.546614 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:22.546715 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.546720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.548147 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.548880 (+ 733us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.550203 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.550214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.552106 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.552109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.552187 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.552208 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 06:44:22.552267 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 06:44:22.552333 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":23,"thread_start_us":93,"threads_started":1}
I20260928 06:44:22.553310 26350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.546053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53302 (local address 127.8.151.193:38055)
0928 06:44:22.546374 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.546389 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.547103 (+ 714us) server_negotiation.cc:407] Connection header received
0928 06:44:22.547207 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.547212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.547386 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.547534 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:22.547894 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.548972 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.550620 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.551437 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.551548 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.551694 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.553090 (+ 1396us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.553171 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":114,"threads_started":1}
I20260928 06:44:22.552721 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.545691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.8.151.194:38427)
0928 06:44:22.546506 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.546516 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.546538 (+ 22us) server_negotiation.cc:407] Connection header received
0928 06:44:22.546610 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.546615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.546783 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.547199 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:22.547344 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.548364 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.550517 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.551368 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.552315 (+ 947us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.552449 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.552499 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.552590 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":134,"threads_started":1}
I20260928 06:44:22.556183 26000 tablet_service.cc:1467] Processing CreateTablet for tablet f3b1494b48604f31b6ea00434039937b (DEFAULT_TABLE table=test-table [id=e4289a83314641b691717e2529789aa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:22.557343 26000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b1494b48604f31b6ea00434039937b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:22.557179 26134 tablet_service.cc:1467] Processing CreateTablet for tablet f3b1494b48604f31b6ea00434039937b (DEFAULT_TABLE table=test-table [id=e4289a83314641b691717e2529789aa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:22.558195 26134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b1494b48604f31b6ea00434039937b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:22.564250 26268 tablet_service.cc:1467] Processing CreateTablet for tablet f3b1494b48604f31b6ea00434039937b (DEFAULT_TABLE table=test-table [id=e4289a83314641b691717e2529789aa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:22.565281 26268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b1494b48604f31b6ea00434039937b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:22.566318 26351 tablet_bootstrap.cc:492] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e: Bootstrap starting.
I20260928 06:44:22.566588 26352 tablet_bootstrap.cc:492] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: Bootstrap starting.
I20260928 06:44:22.569000 26351 tablet_bootstrap.cc:654] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:22.570070 26352 tablet_bootstrap.cc:654] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:22.570385 26351 log.cc:824] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:22.570921 26352 log.cc:824] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:22.572489 26351 tablet_bootstrap.cc:492] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e: No bootstrap required, opened a new log
I20260928 06:44:22.572697 26351 ts_tablet_manager.cc:1397] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260928 06:44:22.572845 26352 tablet_bootstrap.cc:492] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: No bootstrap required, opened a new log
I20260928 06:44:22.573028 26352 ts_tablet_manager.cc:1397] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260928 06:44:22.574052 26355 tablet_bootstrap.cc:492] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: Bootstrap starting.
I20260928 06:44:22.576156 26355 tablet_bootstrap.cc:654] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:22.576494 26351 raft_consensus.cc:348] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 } } peers { permanent_uuid: "c1c8f734815747a0b86fce2a60321f9e" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38055 } } peers { permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 } }
I20260928 06:44:22.576771 26351 raft_consensus.cc:374] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:22.576823 26351 raft_consensus.cc:729] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c8f734815747a0b86fce2a60321f9e, State: Initialized, Role: FOLLOWER
I20260928 06:44:22.577318 26351 consensus_queue.cc:260] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 } } peers { permanent_uuid: "c1c8f734815747a0b86fce2a60321f9e" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38055 } } peers { permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 } }
I20260928 06:44:22.577584 26355 log.cc:824] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:22.577782 26352 raft_consensus.cc:348] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 } } peers { permanent_uuid: "c1c8f734815747a0b86fce2a60321f9e" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38055 } } peers { permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 } }
I20260928 06:44:22.578047 26352 raft_consensus.cc:374] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:22.578111 26352 raft_consensus.cc:729] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 683ed694922d4c9689744abcc545ad99, State: Initialized, Role: FOLLOWER
I20260928 06:44:22.578315 26351 ts_tablet_manager.cc:1428] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260928 06:44:22.578568 26352 consensus_queue.cc:260] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 } } peers { permanent_uuid: "c1c8f734815747a0b86fce2a60321f9e" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38055 } } peers { permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 } }
I20260928 06:44:22.578701 26066 heartbeater.cc:503] Master 127.8.151.254:38957 was elected leader, sending a full tablet report...
I20260928 06:44:22.579217 26200 heartbeater.cc:503] Master 127.8.151.254:38957 was elected leader, sending a full tablet report...
I20260928 06:44:22.579509 26352 ts_tablet_manager.cc:1428] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260928 06:44:22.581005 26355 tablet_bootstrap.cc:492] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: No bootstrap required, opened a new log
I20260928 06:44:22.581203 26355 ts_tablet_manager.cc:1397] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260928 06:44:22.584664 26355 raft_consensus.cc:348] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 } } peers { permanent_uuid: "c1c8f734815747a0b86fce2a60321f9e" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38055 } } peers { permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 } }
I20260928 06:44:22.584877 26355 raft_consensus.cc:374] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:22.584909 26355 raft_consensus.cc:729] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f1d7c34ebb4750a9c6bb5884f96dc6, State: Initialized, Role: FOLLOWER
I20260928 06:44:22.585266 26355 consensus_queue.cc:260] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 } } peers { permanent_uuid: "c1c8f734815747a0b86fce2a60321f9e" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38055 } } peers { permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 } }
I20260928 06:44:22.585871 26334 heartbeater.cc:503] Master 127.8.151.254:38957 was elected leader, sending a full tablet report...
I20260928 06:44:22.586122 26355 ts_tablet_manager.cc:1428] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:44:22.666431 26356 raft_consensus.cc:482] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:44:22.666682 26356 raft_consensus.cc:504] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 } } peers { permanent_uuid: "c1c8f734815747a0b86fce2a60321f9e" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38055 } } peers { permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 } }
I20260928 06:44:22.668179 26356 leader_election.cc:288] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97f1d7c34ebb4750a9c6bb5884f96dc6 (127.8.151.195:37335), 683ed694922d4c9689744abcc545ad99 (127.8.151.194:38427)
I20260928 06:44:22.673071 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.668554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:37833 (local address 127.8.151.195:37335)
0928 06:44:22.668717 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.668723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.669055 (+ 332us) server_negotiation.cc:407] Connection header received
0928 06:44:22.669286 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.669290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.669340 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.669403 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:22.669728 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.670465 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.671799 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.672352 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.672445 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.672498 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.672863 (+ 365us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.672944 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260928 06:44:22.673481 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.668424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:37335 (local address 127.8.151.193:37833)
0928 06:44:22.668932 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:22.668959 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:22.669134 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:22.669447 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:22.669450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:22.669469 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:22.669581 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.669586 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.670611 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.670614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.671630 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.671640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.672607 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.672610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.672717 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.672742 (+ 25us) client_negotiation.cc:769] Sending connection context
0928 06:44:22.673235 (+ 493us) client_negotiation.cc:240] Negotiation successful
0928 06:44:22.673322 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":110,"thread_start_us":89,"threads_started":1}
I20260928 06:44:22.673521 26361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.668424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:38427 (local address 127.8.151.193:54605)
0928 06:44:22.668856 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:22.668872 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:22.669048 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:22.669343 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:22.669346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:22.669363 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:22.669527 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.669534 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.670587 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.670590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.671899 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:22.671912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.672922 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.672926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:22.673049 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.673072 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:44:22.673147 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 06:44:22.673332 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":26,"thread_start_us":132,"threads_started":1}
I20260928 06:44:22.673578 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.668828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:54605 (local address 127.8.151.194:38427)
0928 06:44:22.668950 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.668955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.668970 (+ 15us) server_negotiation.cc:407] Connection header received
0928 06:44:22.669192 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.669195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.669245 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.669319 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:22.669652 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.670477 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.672073 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.672661 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.672764 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.672820 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.673391 (+ 571us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.673470 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:44:22.674190 26288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1494b48604f31b6ea00434039937b" candidate_uuid: "c1c8f734815747a0b86fce2a60321f9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" is_pre_election: true
I20260928 06:44:22.674198 26154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1494b48604f31b6ea00434039937b" candidate_uuid: "c1c8f734815747a0b86fce2a60321f9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "683ed694922d4c9689744abcc545ad99" is_pre_election: true
I20260928 06:44:22.674532 26288 raft_consensus.cc:2454] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1c8f734815747a0b86fce2a60321f9e in term 0.
I20260928 06:44:22.674532 26154 raft_consensus.cc:2454] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1c8f734815747a0b86fce2a60321f9e in term 0.
I20260928 06:44:22.675058 25956 leader_election.cc:302] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 683ed694922d4c9689744abcc545ad99, c1c8f734815747a0b86fce2a60321f9e; no voters:
I20260928 06:44:22.675421 26356 raft_consensus.cc:2781] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:44:22.675544 26356 raft_consensus.cc:482] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:44:22.675598 26356 raft_consensus.cc:3037] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:22.676956 26356 raft_consensus.cc:504] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 } } peers { permanent_uuid: "c1c8f734815747a0b86fce2a60321f9e" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38055 } } peers { permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 } }
I20260928 06:44:22.677383 26356 leader_election.cc:288] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [CANDIDATE]: Term 1 election: Requested vote from peers 97f1d7c34ebb4750a9c6bb5884f96dc6 (127.8.151.195:37335), 683ed694922d4c9689744abcc545ad99 (127.8.151.194:38427)
I20260928 06:44:22.677805 26154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1494b48604f31b6ea00434039937b" candidate_uuid: "c1c8f734815747a0b86fce2a60321f9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "683ed694922d4c9689744abcc545ad99"
I20260928 06:44:22.677961 26154 raft_consensus.cc:3037] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:22.679298 26288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1494b48604f31b6ea00434039937b" candidate_uuid: "c1c8f734815747a0b86fce2a60321f9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6"
I20260928 06:44:22.679350 26154 raft_consensus.cc:2454] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1c8f734815747a0b86fce2a60321f9e in term 1.
I20260928 06:44:22.679448 26288 raft_consensus.cc:3037] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:22.680584 26288 raft_consensus.cc:2454] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1c8f734815747a0b86fce2a60321f9e in term 1.
I20260928 06:44:22.680926 25954 leader_election.cc:302] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97f1d7c34ebb4750a9c6bb5884f96dc6, c1c8f734815747a0b86fce2a60321f9e; no voters:
I20260928 06:44:22.681133 26356 raft_consensus.cc:2781] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:22.681371 26356 raft_consensus.cc:686] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 1 LEADER]: Becoming Leader. State: Replica: c1c8f734815747a0b86fce2a60321f9e, State: Running, Role: LEADER
I20260928 06:44:22.681739 26356 consensus_queue.cc:237] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 } } peers { permanent_uuid: "c1c8f734815747a0b86fce2a60321f9e" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38055 } } peers { permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 } }
I20260928 06:44:22.684787 25874 catalog_manager.cc:5495] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e reported cstate change: term changed from 0 to 1, leader changed from <none> to c1c8f734815747a0b86fce2a60321f9e (127.8.151.193). New cstate: current_term: 1 leader_uuid: "c1c8f734815747a0b86fce2a60321f9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1c8f734815747a0b86fce2a60321f9e" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 } health_report { overall_health: UNKNOWN } } }
I20260928 06:44:22.710832 26350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.707286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.8.151.193:38055)
0928 06:44:22.707439 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.707443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.707457 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:44:22.707567 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.707569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.707629 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.707726 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:22.707995 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.708528 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.709400 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.709625 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.709713 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.710028 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:22.710445 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:22.710568 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.710631 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.710691 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 06:44:22.718349 26154 raft_consensus.cc:1261] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99 [term 1 FOLLOWER]: Refusing update from remote peer c1c8f734815747a0b86fce2a60321f9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:22.718333 26288 raft_consensus.cc:1261] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6 [term 1 FOLLOWER]: Refusing update from remote peer c1c8f734815747a0b86fce2a60321f9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:22.719071 26362 consensus_queue.cc:1035] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "97f1d7c34ebb4750a9c6bb5884f96dc6" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:22.719265 26356 consensus_queue.cc:1035] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "683ed694922d4c9689744abcc545ad99" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 38427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:22.727881 26367 mvcc.cc:205] Tried to move back new op lower bound from 7334206925686231040 to 7334206925547020288. Current Snapshot: MvccSnapshot[applied={T|T < 7334206925686231040}]
I20260928 06:44:22.728072 26365 mvcc.cc:205] Tried to move back new op lower bound from 7334206925686231040 to 7334206925547020288. Current Snapshot: MvccSnapshot[applied={T|T < 7334206925686231040}]
I20260928 06:44:22.730228 26366 mvcc.cc:205] Tried to move back new op lower bound from 7334206925686231040 to 7334206925547020288. Current Snapshot: MvccSnapshot[applied={T|T < 7334206925686231040}]
I20260928 06:44:22.737221 25873 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:22.737411 25873 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:22.740195 25873 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70f410d1c27c4970b3ce08a17ccb5c2b: Sending DeleteTablet for 3 replicas of tablet f3b1494b48604f31b6ea00434039937b
I20260928 06:44:22.741093 26134 tablet_service.cc:1514] Processing DeleteTablet for tablet f3b1494b48604f31b6ea00434039937b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:22 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:53840
I20260928 06:44:22.741318 26000 tablet_service.cc:1514] Processing DeleteTablet for tablet f3b1494b48604f31b6ea00434039937b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:22 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:53302
I20260928 06:44:22.741376 26268 tablet_service.cc:1514] Processing DeleteTablet for tablet f3b1494b48604f31b6ea00434039937b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:22 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:33032
I20260928 06:44:22.742121 26375 tablet_replica.cc:330] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: stopping tablet replica
I20260928 06:44:22.742554 26376 tablet_replica.cc:330] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e: stopping tablet replica
I20260928 06:44:22.742671 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 25937
I20260928 06:44:22.742861 26376 raft_consensus.cc:2229] T f3b1494b48604f31b6ea00434039937b P c1c8f734815747a0b86fce2a60321f9e [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:44:22.742875 26377 tablet_replica.cc:330] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: stopping tablet replica
I20260928 06:44:22.743408 26377 raft_consensus.cc:2229] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:44:22.743814 26377 raft_consensus.cc:2258] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:22.745324 26375 raft_consensus.cc:2229] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:44:22.745481 26377 ts_tablet_manager.cc:1905] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:44:22.745656 26375 raft_consensus.cc:2258] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:22.747212 26375 ts_tablet_manager.cc:1905] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:44:22.748390 26377 ts_tablet_manager.cc:1918] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:44:22.748540 26377 log.cc:1196] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/f3b1494b48604f31b6ea00434039937b
I20260928 06:44:22.748988 26377 ts_tablet_manager.cc:1939] T f3b1494b48604f31b6ea00434039937b P 97f1d7c34ebb4750a9c6bb5884f96dc6: Deleting consensus metadata
I20260928 06:44:22.750211 26375 ts_tablet_manager.cc:1918] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:44:22.750319 26375 log.cc:1196] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/f3b1494b48604f31b6ea00434039937b
I20260928 06:44:22.750424 25860 catalog_manager.cc:4841] TS 97f1d7c34ebb4750a9c6bb5884f96dc6 (127.8.151.195:37335): tablet f3b1494b48604f31b6ea00434039937b (table test-table [id=e4289a83314641b691717e2529789aa8]) successfully deleted
I20260928 06:44:22.750645 26375 ts_tablet_manager.cc:1939] T f3b1494b48604f31b6ea00434039937b P 683ed694922d4c9689744abcc545ad99: Deleting consensus metadata
W20260928 06:44:22.751368 25860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:38055 (error 108)
I20260928 06:44:22.751976 25862 catalog_manager.cc:4841] TS 683ed694922d4c9689744abcc545ad99 (127.8.151.194:38427): tablet f3b1494b48604f31b6ea00434039937b (table test-table [id=e4289a83314641b691717e2529789aa8]) successfully deleted
I20260928 06:44:22.752420 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 26071
I20260928 06:44:22.752468 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.752162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38055 (local address 127.0.0.1:53328)
0928 06:44:22.752302 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:22.752380 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:38055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260928 06:44:22.757627 25860 catalog_manager.cc:4584] TS c1c8f734815747a0b86fce2a60321f9e (127.8.151.193:38055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f3b1494b48604f31b6ea00434039937b: Network error: Client connection negotiation failed: client connection to 127.8.151.193:38055: connect: Connection refused (error 111)
I20260928 06:44:22.757743 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 26205
I20260928 06:44:22.763222 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 25843
2026-09-28T06:44:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2315 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-28T06:44:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:22Z Disabled control of system clock
I20260928 06:44:22.792229 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:45687
--webserver_interface=127.8.151.254
--webserver_port=0
--builtin_ntp_servers=127.8.151.212:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:45687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 06:44:22.900254 26382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:22.900485 26382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:22.900518 26382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:22.904464 26382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:22.904513 26382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:22.904527 26382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:22.904542 26382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:22.904556 26382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:22.909279 26382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:45687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:45687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:22.910270 26382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:22.911646 26382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:22.919533 26387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:22.919533 26388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:22.919786 26382 server_base.cc:1034] running on GCE node
W20260928 06:44:22.919915 26390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:22.920465 26382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:22.921496 26382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:22.922652 26382 hybrid_clock.cc:648] HybridClock initialized: now 1790577862922626 us; error 48 us; skew 500 ppm
I20260928 06:44:22.924392 26382 webserver.cc:460] Webserver started at http://127.8.151.254:44877/ using document root <none> and password file <none>
I20260928 06:44:22.924917 26382 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:22.924983 26382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:22.925163 26382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:22.927116 26382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "7468d2bf25d2f683e3c316487b329765"
server_key_iv: "0fd404503ad0cce61a2c44e6d6a16ba6"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.927673 26382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "7468d2bf25d2f683e3c316487b329765"
server_key_iv: "0fd404503ad0cce61a2c44e6d6a16ba6"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.931246 26382 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260928 06:44:22.933552 26396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:22.934793 26382 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:44:22.934907 26382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd"
format_stamp: "Formatted at 2026-09-28 06:44:22 on dist-test-slave-gtr8"
server_key: "7468d2bf25d2f683e3c316487b329765"
server_key_iv: "0fd404503ad0cce61a2c44e6d6a16ba6"
server_key_version: "encryptionkey@0"
I20260928 06:44:22.934996 26382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:22.951925 26382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:22.952601 26382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:22.952770 26382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:22.959326 26382 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:45687
I20260928 06:44:22.959367 26448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:45687 every 8 connection(s)
I20260928 06:44:22.960246 26382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:22.964000 26449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:22.967406 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 26382
I20260928 06:44:22.967525 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:22.967795 8799 external_mini_cluster.cc:1428] Setting key 5e42f8950ff8dca9c9e93c625118bd4f
I20260928 06:44:22.970956 26449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd: Bootstrap starting.
I20260928 06:44:22.973716 26449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:22.974654 26449 log.cc:824] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:22.976773 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:22.969155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47224 (local address 127.8.151.254:45687)
0928 06:44:22.969770 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:22.969783 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:22.969818 (+ 35us) server_negotiation.cc:407] Connection header received
0928 06:44:22.970406 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:22.970422 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:22.970813 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:22.971203 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:22.971624 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.972567 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.973431 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:22.973949 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:22.974103 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:22.974912 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:22.974918 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:22.974931 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:22.975443 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:22.975516 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:22.975543 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:22.975629 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:22.975773 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 06:44:22.976053 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":116,"threads_started":1}
I20260928 06:44:22.976972 26449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd: No bootstrap required, opened a new log
I20260928 06:44:22.981657 26449 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45687 } }
I20260928 06:44:22.981942 26449 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:22.982004 26449 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac2627fbf1aa43f18a41a27bcb8c03bd, State: Initialized, Role: FOLLOWER
I20260928 06:44:22.982460 26449 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45687 } }
I20260928 06:44:22.982609 26449 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:22.982666 26449 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:22.982756 26449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:22.983913 26449 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45687 } }
I20260928 06:44:22.984310 26449 leader_election.cc:302] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac2627fbf1aa43f18a41a27bcb8c03bd; no voters:
I20260928 06:44:22.984634 26449 leader_election.cc:288] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:22.984740 26454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:22.984956 26454 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [term 1 LEADER]: Becoming Leader. State: Replica: ac2627fbf1aa43f18a41a27bcb8c03bd, State: Running, Role: LEADER
I20260928 06:44:22.985262 26454 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45687 } }
I20260928 06:44:22.985991 26449 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:22.986723 26455 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45687 } } }
I20260928 06:44:22.986846 26455 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:22.987123 26456 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac2627fbf1aa43f18a41a27bcb8c03bd. Latest consensus state: current_term: 1 leader_uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac2627fbf1aa43f18a41a27bcb8c03bd" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 45687 } } }
I20260928 06:44:22.987251 26456 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:22.990541 26469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:22.990607 26469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:22.990684 26461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:22.991551 26461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:22.997539 26461 catalog_manager.cc:1347] Generated new cluster ID: a46a97cd9f384017bbb8ebe96be6c7a0
I20260928 06:44:22.997603 26461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:23.005437 26461 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:23.006599 26461 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:23.020435 26461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd: Generated new TSK 0
I20260928 06:44:23.021136 26461 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:23.029850 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45687
--builtin_ntp_servers=127.8.151.212:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 06:44:23.151736 26473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:23.152108 26473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:23.152184 26473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:23.158260 26473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:23.158344 26473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:23.158430 26473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:44:23.165750 26473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:23.167225 26473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:23.169085 26473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:23.180164 26478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:23.180184 26481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:23.180145 26479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:23.180912 26473 server_base.cc:1034] running on GCE node
I20260928 06:44:23.181429 26473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:23.182233 26473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:23.183401 26473 hybrid_clock.cc:648] HybridClock initialized: now 1790577863183372 us; error 40 us; skew 500 ppm
I20260928 06:44:23.185212 26473 webserver.cc:460] Webserver started at http://127.8.151.193:36197/ using document root <none> and password file <none>
I20260928 06:44:23.185823 26473 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:23.185891 26473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:23.186095 26473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:23.188227 26473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "38a670f475754b9dafb6e9d27e3922cd"
format_stamp: "Formatted at 2026-09-28 06:44:23 on dist-test-slave-gtr8"
server_key: "e73dfc8b3db4db68dfd23bfff8d7cb43"
server_key_iv: "84f6f59c35b101314948b690aaf3d1b1"
server_key_version: "encryptionkey@0"
I20260928 06:44:23.188789 26473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "38a670f475754b9dafb6e9d27e3922cd"
format_stamp: "Formatted at 2026-09-28 06:44:23 on dist-test-slave-gtr8"
server_key: "e73dfc8b3db4db68dfd23bfff8d7cb43"
server_key_iv: "84f6f59c35b101314948b690aaf3d1b1"
server_key_version: "encryptionkey@0"
I20260928 06:44:23.192559 26473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 06:44:23.194994 26487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:23.196317 26473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:44:23.196420 26473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "38a670f475754b9dafb6e9d27e3922cd"
format_stamp: "Formatted at 2026-09-28 06:44:23 on dist-test-slave-gtr8"
server_key: "e73dfc8b3db4db68dfd23bfff8d7cb43"
server_key_iv: "84f6f59c35b101314948b690aaf3d1b1"
server_key_version: "encryptionkey@0"
I20260928 06:44:23.196513 26473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:23.220783 26473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:23.221489 26473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:23.221660 26473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:23.222265 26473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:23.223272 26473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:23.223321 26473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:23.223373 26473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:23.223399 26473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:23.232862 26473 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:44279
I20260928 06:44:23.232895 26600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:44279 every 8 connection(s)
I20260928 06:44:23.233803 26473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:44:23.236249 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 26473
I20260928 06:44:23.236351 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:44:23.236661 8799 external_mini_cluster.cc:1428] Setting key cd17d6a1179ef142f5f811d5d2fde169
I20260928 06:44:23.239024 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45687
--builtin_ntp_servers=127.8.151.212:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 06:44:23.245081 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.235692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:35403 (local address 127.8.151.254:45687)
0928 06:44:23.235811 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:23.235815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:23.237301 (+ 1486us) server_negotiation.cc:407] Connection header received
0928 06:44:23.238981 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:23.238984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:23.239039 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:23.239110 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:23.240219 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.240944 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.242711 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.243066 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.243211 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.244542 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:23.244546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:23.244549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:23.244563 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:23.244588 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:23.244599 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:23.244677 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:23.244855 (+ 178us) server_negotiation.cc:299] Negotiation successful
0928 06:44:23.244923 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 06:44:23.246141 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.236255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45687 (local address 127.8.151.193:35403)
0928 06:44:23.237126 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:23.237185 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:23.238759 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:23.239469 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:23.239477 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:23.239565 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:23.240068 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.240081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.241093 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.241097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.242532 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.242544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.243255 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.243260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.243472 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.244214 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:44:23.244216 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:23.244275 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:44:23.244360 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:23.244693 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:23.244702 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:23.244709 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 06:44:23.244849 (+ 140us) client_negotiation.cc:240] Negotiation successful
0928 06:44:23.245153 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":142,"threads_started":1}
I20260928 06:44:23.247400 26601 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45687
I20260928 06:44:23.247666 26601 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:23.248311 26601 heartbeater.cc:511] Master 127.8.151.254:45687 requested a full tablet report, sending...
I20260928 06:44:23.250030 26413 ts_manager.cc:194] Registered new tserver with Master: 38a670f475754b9dafb6e9d27e3922cd (127.8.151.193:44279)
I20260928 06:44:23.251580 26413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.193:35403
W20260928 06:44:23.363521 26604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:23.363775 26604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:23.363812 26604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:23.367838 26604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:23.367887 26604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:23.367939 26604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:44:23.372632 26604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:23.373693 26604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:23.375048 26604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:23.385329 26612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:23.385353 26609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:23.385353 26610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:23.385767 26604 server_base.cc:1034] running on GCE node
I20260928 06:44:23.386222 26604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:23.387023 26604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:23.388211 26604 hybrid_clock.cc:648] HybridClock initialized: now 1790577863388185 us; error 37 us; skew 500 ppm
I20260928 06:44:23.390367 26604 webserver.cc:460] Webserver started at http://127.8.151.194:36391/ using document root <none> and password file <none>
I20260928 06:44:23.391129 26604 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:23.391244 26604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:23.391487 26604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:23.394170 26604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "bc0e43f34bd2403ca9de3846d392f740"
format_stamp: "Formatted at 2026-09-28 06:44:23 on dist-test-slave-gtr8"
server_key: "139c43251cd243477482b45cd7790e71"
server_key_iv: "e8df1f6451bac16d4c3f657a43077348"
server_key_version: "encryptionkey@0"
I20260928 06:44:23.394852 26604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "bc0e43f34bd2403ca9de3846d392f740"
format_stamp: "Formatted at 2026-09-28 06:44:23 on dist-test-slave-gtr8"
server_key: "139c43251cd243477482b45cd7790e71"
server_key_iv: "e8df1f6451bac16d4c3f657a43077348"
server_key_version: "encryptionkey@0"
I20260928 06:44:23.398715 26604 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:44:23.401391 26618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:23.402738 26604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:44:23.402841 26604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "bc0e43f34bd2403ca9de3846d392f740"
format_stamp: "Formatted at 2026-09-28 06:44:23 on dist-test-slave-gtr8"
server_key: "139c43251cd243477482b45cd7790e71"
server_key_iv: "e8df1f6451bac16d4c3f657a43077348"
server_key_version: "encryptionkey@0"
I20260928 06:44:23.402940 26604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:23.425937 26604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:23.426682 26604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:23.426855 26604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:23.427490 26604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:23.428435 26604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:23.428480 26604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:23.428519 26604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:23.428547 26604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:23.438401 26604 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:36997
I20260928 06:44:23.438434 26733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:36997 every 8 connection(s)
I20260928 06:44:23.439443 26604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:44:23.446545 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 26604
I20260928 06:44:23.446687 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:44:23.447028 8799 external_mini_cluster.cc:1428] Setting key 39b6690f36f8696d5ea89e76fd53245b
I20260928 06:44:23.449191 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45687
--builtin_ntp_servers=127.8.151.212:43289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 06:44:23.450373 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.441419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:59159 (local address 127.8.151.254:45687)
0928 06:44:23.441584 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:23.441589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:23.442730 (+ 1141us) server_negotiation.cc:407] Connection header received
0928 06:44:23.444514 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:23.444518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:23.444575 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:23.444797 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:23.445696 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.446418 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.448037 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.448257 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.448352 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.449854 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:23.449859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:23.449862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:23.449877 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:23.449906 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:23.449917 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:23.449994 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:23.450150 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 06:44:23.450216 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:44:23.451327 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.441817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45687 (local address 127.8.151.194:59159)
0928 06:44:23.442543 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:23.442614 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:23.444292 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:23.444978 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:23.444985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:23.445070 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:23.445553 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.445566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.446574 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.446578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.447923 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.447934 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.448517 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.448520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.448716 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.449540 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:44:23.449542 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:23.449600 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:44:23.449683 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:23.450002 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:23.450009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:23.450014 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 06:44:23.450145 (+ 131us) client_negotiation.cc:240] Negotiation successful
0928 06:44:23.450445 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":115,"threads_started":1}
I20260928 06:44:23.452394 26734 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45687
I20260928 06:44:23.452620 26734 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:23.453127 26734 heartbeater.cc:511] Master 127.8.151.254:45687 requested a full tablet report, sending...
I20260928 06:44:23.454085 26413 ts_manager.cc:194] Registered new tserver with Master: bc0e43f34bd2403ca9de3846d392f740 (127.8.151.194:36997)
I20260928 06:44:23.454635 26413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.194:59159
W20260928 06:44:23.590680 26737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:23.591022 26737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:23.591075 26737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:23.597009 26737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:23.597091 26737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:23.597172 26737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:44:23.603986 26737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:45687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:23.605264 26737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:23.606907 26737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:23.617787 26742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:23.617797 26743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:23.618207 26745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:23.618381 26737 server_base.cc:1034] running on GCE node
I20260928 06:44:23.618772 26737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:23.619524 26737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:23.620705 26737 hybrid_clock.cc:648] HybridClock initialized: now 1790577863620686 us; error 38 us; skew 500 ppm
I20260928 06:44:23.622699 26737 webserver.cc:460] Webserver started at http://127.8.151.195:33119/ using document root <none> and password file <none>
I20260928 06:44:23.623301 26737 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:23.623370 26737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:23.623596 26737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:23.626045 26737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "4ae11485bcc9489ea2dc02ec8b82d397"
format_stamp: "Formatted at 2026-09-28 06:44:23 on dist-test-slave-gtr8"
server_key: "3cc209b800144a51cbaca6fffffdd2ca"
server_key_iv: "1a1fd0661e8e108e87eb5613db5fd7ca"
server_key_version: "encryptionkey@0"
I20260928 06:44:23.627384 26737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "4ae11485bcc9489ea2dc02ec8b82d397"
format_stamp: "Formatted at 2026-09-28 06:44:23 on dist-test-slave-gtr8"
server_key: "3cc209b800144a51cbaca6fffffdd2ca"
server_key_iv: "1a1fd0661e8e108e87eb5613db5fd7ca"
server_key_version: "encryptionkey@0"
I20260928 06:44:23.631726 26737 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260928 06:44:23.634315 26751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:23.635807 26737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:23.635907 26737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "4ae11485bcc9489ea2dc02ec8b82d397"
format_stamp: "Formatted at 2026-09-28 06:44:23 on dist-test-slave-gtr8"
server_key: "3cc209b800144a51cbaca6fffffdd2ca"
server_key_iv: "1a1fd0661e8e108e87eb5613db5fd7ca"
server_key_version: "encryptionkey@0"
I20260928 06:44:23.636035 26737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:23.650797 26737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:23.651481 26737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:23.651646 26737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:23.652197 26737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:23.653080 26737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:23.653124 26737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:23.653175 26737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:23.653200 26737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:23.662609 26737 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:42211
I20260928 06:44:23.662647 26865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:42211 every 8 connection(s)
I20260928 06:44:23.663812 26737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:44:23.666039 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 26737
I20260928 06:44:23.666141 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:44:23.666404 8799 external_mini_cluster.cc:1428] Setting key 16e823922a3e607be1868cd5d5d7f8e0
I20260928 06:44:23.673386 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.665691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:36153 (local address 127.8.151.254:45687)
0928 06:44:23.665828 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:23.665833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:23.667354 (+ 1521us) server_negotiation.cc:407] Connection header received
0928 06:44:23.668963 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:23.668966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:23.669021 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:23.669093 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:23.669970 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.670489 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.671645 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.671869 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.671955 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.672951 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:23.672955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:23.672957 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:23.672972 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:23.672989 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:23.672997 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:23.673053 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:23.673186 (+ 133us) server_negotiation.cc:299] Negotiation successful
0928 06:44:23.673244 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 06:44:23.674257 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.666212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:45687 (local address 127.8.151.195:36153)
0928 06:44:23.667217 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:23.667272 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:23.668776 (+ 1504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:23.669278 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:23.669285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:23.669367 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:23.669842 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.669854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.670603 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.670607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.671535 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.671544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.672001 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.672005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.672153 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.672710 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:44:23.672711 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:23.672751 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:44:23.672813 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:23.673063 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:23.673072 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:23.673078 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:44:23.673178 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 06:44:23.673453 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":130,"threads_started":1}
I20260928 06:44:23.675359 26866 heartbeater.cc:347] Connected to a master server at 127.8.151.254:45687
I20260928 06:44:23.675571 26866 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:23.676149 26866 heartbeater.cc:511] Master 127.8.151.254:45687 requested a full tablet report, sending...
I20260928 06:44:23.677218 26413 ts_manager.cc:194] Registered new tserver with Master: 4ae11485bcc9489ea2dc02ec8b82d397 (127.8.151.195:42211)
I20260928 06:44:23.677805 26413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.195:36153
I20260928 06:44:23.680207 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:44:23.686162 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.682139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.8.151.254:45687)
0928 06:44:23.682249 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:23.682252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:23.682462 (+ 210us) server_negotiation.cc:407] Connection header received
0928 06:44:23.682573 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:23.682576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:23.682619 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:23.682683 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:23.682927 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.683494 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.684733 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.684948 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.685029 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.685799 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:23.685803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:23.685804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:23.685815 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:23.685832 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:23.685840 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:23.685894 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:23.685988 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 06:44:23.686037 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 06:44:23.688922 26413 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:44:23.692240 26413 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:44:23.707818 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.701061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:36997 (local address 127.0.0.1:36030)
0928 06:44:23.701401 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:23.701447 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:23.701566 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:23.702664 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:23.702670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:23.702694 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:23.702853 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.702861 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.704237 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.704240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.706095 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.706103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.707208 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.707213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.707341 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.707368 (+ 27us) client_negotiation.cc:769] Sending connection context
0928 06:44:23.707451 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 06:44:23.707635 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":431,"thread_start_us":91,"threads_started":1}
I20260928 06:44:23.707819 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.701314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:44279 (local address 127.0.0.1:45838)
0928 06:44:23.701533 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:23.701546 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:23.701634 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:23.702753 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:23.702756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:23.702769 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:23.703002 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.703008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.704414 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.704417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.705711 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.705720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.707203 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.707206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.707315 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.707364 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 06:44:23.707454 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 06:44:23.707643 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":124,"thread_start_us":86,"threads_started":1}
I20260928 06:44:23.708655 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.701892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45838 (local address 127.8.151.193:44279)
0928 06:44:23.702155 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:23.702162 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:23.702187 (+ 25us) server_negotiation.cc:407] Connection header received
0928 06:44:23.702263 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:23.702268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:23.702469 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:23.702625 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:23.703116 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.704302 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.706226 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.707046 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.708127 (+ 1081us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.708334 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:23.708381 (+ 47us) server_negotiation.cc:299] Negotiation successful
0928 06:44:23.708481 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":88,"threads_started":1}
I20260928 06:44:23.709407 26879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.701870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36030 (local address 127.8.151.194:36997)
0928 06:44:23.702164 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:23.702175 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:23.702196 (+ 21us) server_negotiation.cc:407] Connection header received
0928 06:44:23.702258 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:23.702263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:23.702466 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:23.702849 (+ 383us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:23.702987 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.704125 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.706207 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.707039 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.708180 (+ 1141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.708362 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:23.708411 (+ 49us) server_negotiation.cc:299] Negotiation successful
0928 06:44:23.709241 (+ 830us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":104,"threads_started":1}
I20260928 06:44:23.710844 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.702324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:42211 (local address 127.0.0.1:57060)
0928 06:44:23.703427 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:23.703440 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:23.703572 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:23.704144 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:23.704146 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:23.704163 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:23.705445 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.705456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.706912 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.706915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.708459 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.708469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.710499 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.710501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.710567 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.710584 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 06:44:23.710642 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 06:44:23.710714 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"mutex_wait_us":16,"thread_start_us":95,"threads_started":1}
I20260928 06:44:23.710945 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.702483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57060 (local address 127.8.151.195:42211)
0928 06:44:23.703290 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:23.703304 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:23.703510 (+ 206us) server_negotiation.cc:407] Connection header received
0928 06:44:23.703705 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:23.703711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:23.703872 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:23.703974 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:23.705581 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.706772 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.708593 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.710233 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.710296 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.710421 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:23.710698 (+ 277us) server_negotiation.cc:299] Negotiation successful
0928 06:44:23.710793 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":98,"threads_started":1}
I20260928 06:44:23.711790 26535 tablet_service.cc:1467] Processing CreateTablet for tablet e49b02a655dd45d5a612d544b4fe4a81 (DEFAULT_TABLE table=test-table [id=ea32596bccf146ee93e9b7ecc3aef924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:23.712092 26666 tablet_service.cc:1467] Processing CreateTablet for tablet e49b02a655dd45d5a612d544b4fe4a81 (DEFAULT_TABLE table=test-table [id=ea32596bccf146ee93e9b7ecc3aef924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:23.712994 26666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e49b02a655dd45d5a612d544b4fe4a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:23.712994 26535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e49b02a655dd45d5a612d544b4fe4a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:23.713874 26800 tablet_service.cc:1467] Processing CreateTablet for tablet e49b02a655dd45d5a612d544b4fe4a81 (DEFAULT_TABLE table=test-table [id=ea32596bccf146ee93e9b7ecc3aef924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:23.714851 26800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e49b02a655dd45d5a612d544b4fe4a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:23.721491 26883 tablet_bootstrap.cc:492] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd: Bootstrap starting.
I20260928 06:44:23.723228 26885 tablet_bootstrap.cc:492] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: Bootstrap starting.
I20260928 06:44:23.723589 26884 tablet_bootstrap.cc:492] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: Bootstrap starting.
I20260928 06:44:23.723973 26883 tablet_bootstrap.cc:654] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:23.724843 26883 log.cc:824] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:23.725083 26885 tablet_bootstrap.cc:654] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:23.725739 26884 tablet_bootstrap.cc:654] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:23.725800 26885 log.cc:824] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:23.726541 26884 log.cc:824] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:23.726894 26883 tablet_bootstrap.cc:492] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd: No bootstrap required, opened a new log
I20260928 06:44:23.727083 26883 ts_tablet_manager.cc:1397] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 06:44:23.728469 26884 tablet_bootstrap.cc:492] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: No bootstrap required, opened a new log
I20260928 06:44:23.728644 26884 ts_tablet_manager.cc:1397] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260928 06:44:23.728650 26885 tablet_bootstrap.cc:492] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: No bootstrap required, opened a new log
I20260928 06:44:23.728811 26885 ts_tablet_manager.cc:1397] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260928 06:44:23.731402 26883 raft_consensus.cc:348] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 } } peers { permanent_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 42211 } } peers { permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 } }
I20260928 06:44:23.731669 26883 raft_consensus.cc:374] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:23.731724 26883 raft_consensus.cc:729] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a670f475754b9dafb6e9d27e3922cd, State: Initialized, Role: FOLLOWER
I20260928 06:44:23.731992 26884 raft_consensus.cc:348] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 } } peers { permanent_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 42211 } } peers { permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 } }
I20260928 06:44:23.732199 26884 raft_consensus.cc:374] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:23.732250 26884 raft_consensus.cc:729] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc0e43f34bd2403ca9de3846d392f740, State: Initialized, Role: FOLLOWER
I20260928 06:44:23.732434 26883 consensus_queue.cc:260] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 } } peers { permanent_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 42211 } } peers { permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 } }
I20260928 06:44:23.732671 26884 consensus_queue.cc:260] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 } } peers { permanent_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 42211 } } peers { permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 } }
I20260928 06:44:23.732757 26885 raft_consensus.cc:348] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 } } peers { permanent_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 42211 } } peers { permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 } }
I20260928 06:44:23.732990 26885 raft_consensus.cc:374] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:23.733031 26885 raft_consensus.cc:729] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae11485bcc9489ea2dc02ec8b82d397, State: Initialized, Role: FOLLOWER
I20260928 06:44:23.733448 26883 ts_tablet_manager.cc:1428] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260928 06:44:23.733404 26885 consensus_queue.cc:260] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 } } peers { permanent_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 42211 } } peers { permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 } }
I20260928 06:44:23.733608 26884 ts_tablet_manager.cc:1428] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:44:23.733795 26601 heartbeater.cc:503] Master 127.8.151.254:45687 was elected leader, sending a full tablet report...
I20260928 06:44:23.733909 26734 heartbeater.cc:503] Master 127.8.151.254:45687 was elected leader, sending a full tablet report...
I20260928 06:44:23.734261 26885 ts_tablet_manager.cc:1428] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260928 06:44:23.734285 26866 heartbeater.cc:503] Master 127.8.151.254:45687 was elected leader, sending a full tablet report...
W20260928 06:44:23.735085 26602 tablet.cc:2369] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:44:23.800978 26891 raft_consensus.cc:482] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:44:23.801219 26891 raft_consensus.cc:504] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 } } peers { permanent_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 42211 } } peers { permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 } }
I20260928 06:44:23.802394 26891 leader_election.cc:288] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc0e43f34bd2403ca9de3846d392f740 (127.8.151.194:36997), 38a670f475754b9dafb6e9d27e3922cd (127.8.151.193:44279)
I20260928 06:44:23.806831 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.802633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:36997 (local address 127.8.151.195:56907)
0928 06:44:23.802931 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:23.802947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:23.803048 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:23.803347 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:23.803350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:23.803366 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:23.803473 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.803478 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.804273 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.804276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.805351 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.805361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.806331 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.806335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.806455 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.806507 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 06:44:23.806565 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 06:44:23.806644 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260928 06:44:23.807125 26879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.802765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:56907 (local address 127.8.151.194:36997)
0928 06:44:23.803122 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:23.803127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:23.803164 (+ 37us) server_negotiation.cc:407] Connection header received
0928 06:44:23.803216 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:23.803219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:23.803262 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:23.803326 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:23.803615 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.804166 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.805504 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.806159 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.806819 (+ 660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.806872 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:23.806914 (+ 42us) server_negotiation.cc:299] Negotiation successful
0928 06:44:23.806991 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 06:44:23.807245 26892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.802711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:44279 (local address 127.8.151.195:58777)
0928 06:44:23.803075 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:23.803090 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:23.803227 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:23.803546 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:23.803549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:23.803566 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:23.803673 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.803679 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.804712 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.804716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.805691 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:23.805698 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.806369 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.806373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:23.806485 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.806517 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 06:44:23.806633 (+ 116us) client_negotiation.cc:240] Negotiation successful
0928 06:44:23.806704 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":5,"thread_start_us":136,"threads_started":1}
I20260928 06:44:23.807634 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.802900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:58777 (local address 127.8.151.193:44279)
0928 06:44:23.803022 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:23.803026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:23.803186 (+ 160us) server_negotiation.cc:407] Connection header received
0928 06:44:23.803373 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:23.803377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:23.803433 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:23.803525 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:23.803781 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.804587 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.805798 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.806247 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.806361 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.806414 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:23.807452 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0928 06:44:23.807540 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 06:44:23.807788 26687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49b02a655dd45d5a612d544b4fe4a81" candidate_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc0e43f34bd2403ca9de3846d392f740" is_pre_election: true
I20260928 06:44:23.808135 26687 raft_consensus.cc:2454] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ae11485bcc9489ea2dc02ec8b82d397 in term 0.
I20260928 06:44:23.808153 26555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49b02a655dd45d5a612d544b4fe4a81" candidate_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a670f475754b9dafb6e9d27e3922cd" is_pre_election: true
I20260928 06:44:23.808411 26555 raft_consensus.cc:2454] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ae11485bcc9489ea2dc02ec8b82d397 in term 0.
I20260928 06:44:23.808725 26755 leader_election.cc:302] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ae11485bcc9489ea2dc02ec8b82d397, bc0e43f34bd2403ca9de3846d392f740; no voters:
I20260928 06:44:23.809031 26891 raft_consensus.cc:2781] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:44:23.809195 26891 raft_consensus.cc:482] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:44:23.809305 26891 raft_consensus.cc:3037] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:23.810309 26891 raft_consensus.cc:504] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 } } peers { permanent_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 42211 } } peers { permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 } }
I20260928 06:44:23.810652 26891 leader_election.cc:288] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [CANDIDATE]: Term 1 election: Requested vote from peers bc0e43f34bd2403ca9de3846d392f740 (127.8.151.194:36997), 38a670f475754b9dafb6e9d27e3922cd (127.8.151.193:44279)
I20260928 06:44:23.811030 26687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49b02a655dd45d5a612d544b4fe4a81" candidate_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc0e43f34bd2403ca9de3846d392f740"
I20260928 06:44:23.811053 26555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49b02a655dd45d5a612d544b4fe4a81" candidate_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a670f475754b9dafb6e9d27e3922cd"
I20260928 06:44:23.811167 26687 raft_consensus.cc:3037] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:23.811203 26555 raft_consensus.cc:3037] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:23.812146 26687 raft_consensus.cc:2454] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ae11485bcc9489ea2dc02ec8b82d397 in term 1.
I20260928 06:44:23.812412 26755 leader_election.cc:302] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ae11485bcc9489ea2dc02ec8b82d397, bc0e43f34bd2403ca9de3846d392f740; no voters:
I20260928 06:44:23.812456 26555 raft_consensus.cc:2454] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ae11485bcc9489ea2dc02ec8b82d397 in term 1.
I20260928 06:44:23.812575 26891 raft_consensus.cc:2781] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:23.812772 26891 raft_consensus.cc:686] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 1 LEADER]: Becoming Leader. State: Replica: 4ae11485bcc9489ea2dc02ec8b82d397, State: Running, Role: LEADER
I20260928 06:44:23.813113 26891 consensus_queue.cc:237] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 } } peers { permanent_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 42211 } } peers { permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 } }
I20260928 06:44:23.816131 26411 catalog_manager.cc:5495] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ae11485bcc9489ea2dc02ec8b82d397 (127.8.151.195). New cstate: current_term: 1 leader_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ae11485bcc9489ea2dc02ec8b82d397" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 42211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 } health_report { overall_health: UNKNOWN } } }
I20260928 06:44:23.842754 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.838259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57068 (local address 127.8.151.195:42211)
0928 06:44:23.838407 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:23.838413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:23.838431 (+ 18us) server_negotiation.cc:407] Connection header received
0928 06:44:23.838534 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:23.838537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:23.838612 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:23.838737 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:23.839048 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.839913 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.840948 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:23.841298 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:23.841417 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:23.841707 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:23.842256 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:23.842392 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:23.842500 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 06:44:23.842564 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 06:44:23.849555 26687 raft_consensus.cc:1261] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740 [term 1 FOLLOWER]: Refusing update from remote peer 4ae11485bcc9489ea2dc02ec8b82d397: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:23.849555 26555 raft_consensus.cc:1261] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd [term 1 FOLLOWER]: Refusing update from remote peer 4ae11485bcc9489ea2dc02ec8b82d397: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:23.850368 26891 consensus_queue.cc:1035] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a670f475754b9dafb6e9d27e3922cd" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 44279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:23.850608 26893 consensus_queue.cc:1035] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc0e43f34bd2403ca9de3846d392f740" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:23.863332 26899 mvcc.cc:205] Tried to move back new op lower bound from 7334206930319863808 to 7334206930181222400. Current Snapshot: MvccSnapshot[applied={T|T < 7334206930319863808}]
I20260928 06:44:23.871765 26412 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:23.871968 26412 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:23.875056 26412 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac2627fbf1aa43f18a41a27bcb8c03bd: Sending DeleteTablet for 3 replicas of tablet e49b02a655dd45d5a612d544b4fe4a81
I20260928 06:44:23.876072 26535 tablet_service.cc:1514] Processing DeleteTablet for tablet e49b02a655dd45d5a612d544b4fe4a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:23 UTC) from {username='slave'} at 127.0.0.1:45838
I20260928 06:44:23.876127 26800 tablet_service.cc:1514] Processing DeleteTablet for tablet e49b02a655dd45d5a612d544b4fe4a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:23 UTC) from {username='slave'} at 127.0.0.1:57060
I20260928 06:44:23.876299 26666 tablet_service.cc:1514] Processing DeleteTablet for tablet e49b02a655dd45d5a612d544b4fe4a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:23 UTC) from {username='slave'} at 127.0.0.1:36030
I20260928 06:44:23.876773 26907 tablet_replica.cc:330] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: stopping tablet replica
I20260928 06:44:23.876791 26905 tablet_replica.cc:330] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd: stopping tablet replica
I20260928 06:44:23.877111 26905 raft_consensus.cc:2229] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:44:23.877123 26907 raft_consensus.cc:2229] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:44:23.877449 26905 raft_consensus.cc:2258] T e49b02a655dd45d5a612d544b4fe4a81 P 38a670f475754b9dafb6e9d27e3922cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:23.877450 26907 raft_consensus.cc:2258] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:23.878043 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 26473
I20260928 06:44:23.879021 26907 ts_tablet_manager.cc:1905] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:44:23.881940 26907 ts_tablet_manager.cc:1918] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:44:23.882054 26907 log.cc:1196] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/e49b02a655dd45d5a612d544b4fe4a81
I20260928 06:44:23.882390 26907 ts_tablet_manager.cc:1939] T e49b02a655dd45d5a612d544b4fe4a81 P bc0e43f34bd2403ca9de3846d392f740: Deleting consensus metadata
I20260928 06:44:23.883863 26400 catalog_manager.cc:4841] TS bc0e43f34bd2403ca9de3846d392f740 (127.8.151.194:36997): tablet e49b02a655dd45d5a612d544b4fe4a81 (table test-table [id=ea32596bccf146ee93e9b7ecc3aef924]) successfully deleted
I20260928 06:44:23.884418 26906 tablet_replica.cc:330] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: stopping tablet replica
I20260928 06:44:23.884783 26906 raft_consensus.cc:2229] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:44:23.885277 26906 raft_consensus.cc:2258] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:23.887070 26906 ts_tablet_manager.cc:1905] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 06:44:23.889284 26397 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:44279 (error 108)
I20260928 06:44:23.890120 26906 ts_tablet_manager.cc:1918] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:44:23.890162 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 26604
I20260928 06:44:23.890813 26906 log.cc:1196] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/e49b02a655dd45d5a612d544b4fe4a81
I20260928 06:44:23.891266 26906 ts_tablet_manager.cc:1939] T e49b02a655dd45d5a612d544b4fe4a81 P 4ae11485bcc9489ea2dc02ec8b82d397: Deleting consensus metadata
I20260928 06:44:23.892560 26399 catalog_manager.cc:4841] TS 4ae11485bcc9489ea2dc02ec8b82d397 (127.8.151.195:42211): tablet e49b02a655dd45d5a612d544b4fe4a81 (table test-table [id=ea32596bccf146ee93e9b7ecc3aef924]) successfully deleted
I20260928 06:44:23.894635 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:23.894272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:44279 (local address 127.0.0.1:45846)
0928 06:44:23.894457 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:23.894541 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:44279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260928 06:44:23.894807 26397 catalog_manager.cc:4584] TS 38a670f475754b9dafb6e9d27e3922cd (127.8.151.193:44279): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e49b02a655dd45d5a612d544b4fe4a81: Network error: Client connection negotiation failed: client connection to 127.8.151.193:44279: connect: Connection refused (error 111)
I20260928 06:44:23.898509 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 26737
I20260928 06:44:23.903714 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 26382
2026-09-28T06:44:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1141 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:23 dist-test-slave-gtr8 krb5kdc[26911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:23 dist-test-slave-gtr8 krb5kdc[26911](info): set up 2 sockets
Sep 28 06:44:23 dist-test-slave-gtr8 krb5kdc[26911](info): commencing operation
krb5kdc: starting...
W20260928 06:44:24.903364 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:24 dist-test-slave-gtr8 krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:25.022275 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:41559
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:41559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:25.128185 26925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:25.128469 26925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:25.128527 26925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:25.132565 26925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:25.132628 26925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:25.132655 26925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:25.132678 26925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:25.132706 26925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:25.138005 26925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:41559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:41559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:25.139073 26925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:25.140103 26925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:25.148196 26930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:25.148468 26925 server_base.cc:1034] running on GCE node
W20260928 06:44:25.148413 26931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:25.148188 26933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:25.149139 26925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:25.150206 26925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:25.151360 26925 hybrid_clock.cc:648] HybridClock initialized: now 1790577865151336 us; error 42 us; skew 500 ppm
Sep 28 06:44:25 dist-test-slave-gtr8 krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:25.154135 26925 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:25.155027 26925 webserver.cc:460] Webserver started at http://127.8.151.254:34273/ using document root <none> and password file <none>
I20260928 06:44:25.155629 26925 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:25.155702 26925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:25.155896 26925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:25.157964 26925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9"
format_stamp: "Formatted at 2026-09-28 06:44:25 on dist-test-slave-gtr8"
server_key: "cbdf280651124defd7bd8d993982f506"
server_key_iv: "d791017b978016fd737c32b3fc1ae2d3"
server_key_version: "encryptionkey@0"
I20260928 06:44:25.158560 26925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9"
format_stamp: "Formatted at 2026-09-28 06:44:25 on dist-test-slave-gtr8"
server_key: "cbdf280651124defd7bd8d993982f506"
server_key_iv: "d791017b978016fd737c32b3fc1ae2d3"
server_key_version: "encryptionkey@0"
I20260928 06:44:25.162345 26925 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:44:25.164934 26940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:25.166204 26925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:44:25.166322 26925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9"
format_stamp: "Formatted at 2026-09-28 06:44:25 on dist-test-slave-gtr8"
server_key: "cbdf280651124defd7bd8d993982f506"
server_key_iv: "d791017b978016fd737c32b3fc1ae2d3"
server_key_version: "encryptionkey@0"
I20260928 06:44:25.166419 26925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:25.187938 26925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:25.191124 26925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:25.191424 26925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:25.198745 26925 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:41559
I20260928 06:44:25.198762 26992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:41559 every 8 connection(s)
I20260928 06:44:25.199752 26925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:25.202453 26993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:25.208266 26993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9: Bootstrap starting.
I20260928 06:44:25.208336 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 26925
I20260928 06:44:25.208457 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:25.208778 8799 external_mini_cluster.cc:1428] Setting key e1f5022c7b3867c5fd97a7b313a8df2c
I20260928 06:44:25.211002 26993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:25.211740 26993 log.cc:824] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:25.213793 26993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9: No bootstrap required, opened a new log
I20260928 06:44:25.217562 26993 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41559 } }
I20260928 06:44:25.217820 26993 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:25.217877 26993 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd64bbbe6ed34e01aeecc834a2ec3ca9, State: Initialized, Role: FOLLOWER
I20260928 06:44:25.218375 26993 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41559 } }
I20260928 06:44:25.218528 26993 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:25.218590 26993 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:25.218688 26993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [term 0 FOLLOWER]: Advancing to term 1
Sep 28 06:44:25 dist-test-slave-gtr8 krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:25.219812 26993 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41559 } }
I20260928 06:44:25.220161 26993 leader_election.cc:302] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd64bbbe6ed34e01aeecc834a2ec3ca9; no voters:
I20260928 06:44:25.220477 26993 leader_election.cc:288] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:25.220841 26998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:25.221029 26998 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [term 1 LEADER]: Becoming Leader. State: Replica: bd64bbbe6ed34e01aeecc834a2ec3ca9, State: Running, Role: LEADER
I20260928 06:44:25.221550 26998 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41559 } }
I20260928 06:44:25.221813 26993 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:25.222702 26998 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd64bbbe6ed34e01aeecc834a2ec3ca9. Latest consensus state: current_term: 1 leader_uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41559 } } }
I20260928 06:44:25.222824 26998 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:25.223910 27007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:25.226123 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:25.210223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47766 (local address 127.8.151.254:41559)
0928 06:44:25.210732 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:25.210751 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:25.210922 (+ 171us) server_negotiation.cc:407] Connection header received
0928 06:44:25.211903 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:25.211927 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:25.212361 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:25.212776 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:25.213578 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:25.215093 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:25.216321 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:25.216999 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:25.217151 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:25.220106 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:25.220112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:25.220130 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:25.220160 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:25.222175 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:25.222752 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:25.222758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:25.222763 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:25.222899 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:25.223302 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:25.223305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:25.223306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:25.223791 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:25.223981 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:25.224844 (+ 863us) server_negotiation.cc:299] Negotiation successful
0928 06:44:25.225133 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":148,"threads_started":1}
I20260928 06:44:25.226999 27007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:25.227092 26999 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd64bbbe6ed34e01aeecc834a2ec3ca9" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 41559 } } }
I20260928 06:44:25.227363 26999 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:25.232764 27007 catalog_manager.cc:1347] Generated new cluster ID: e0d701558de742a2abd86bca43c08a9e
I20260928 06:44:25.232827 27007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:25.264741 27007 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:25.265972 27007 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:25.270834 27007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9: Generated new TSK 0
I20260928 06:44:25.271607 27007 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:44:25.309774 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41559
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:25.462190 27019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:25.462543 27019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:25.462615 27019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:25.468711 27019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:25.468796 27019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:25.468883 27019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:44:25.476818 27019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:25.478168 27019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:25.479512 27019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:25.491308 27025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:25.491308 27024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:25.491312 27027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:25.491658 27019 server_base.cc:1034] running on GCE node
I20260928 06:44:25.492094 27019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:25.492858 27019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:25.494024 27019 hybrid_clock.cc:648] HybridClock initialized: now 1790577865493949 us; error 93 us; skew 500 ppm
Sep 28 06:44:25 dist-test-slave-gtr8 krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:25.503021 27019 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:44:25.504169 27019 webserver.cc:460] Webserver started at http://127.8.151.193:41365/ using document root <none> and password file <none>
I20260928 06:44:25.504881 27019 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:25.504974 27019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:25.505219 27019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:25.507990 27019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "35215848daac4ff3865433cf1c666fb0"
format_stamp: "Formatted at 2026-09-28 06:44:25 on dist-test-slave-gtr8"
server_key: "85a5d8ee9229167443e671906c8e67ec"
server_key_iv: "950644d52fe50dab79bb2be83c4e2504"
server_key_version: "encryptionkey@0"
I20260928 06:44:25.508662 27019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "35215848daac4ff3865433cf1c666fb0"
format_stamp: "Formatted at 2026-09-28 06:44:25 on dist-test-slave-gtr8"
server_key: "85a5d8ee9229167443e671906c8e67ec"
server_key_iv: "950644d52fe50dab79bb2be83c4e2504"
server_key_version: "encryptionkey@0"
I20260928 06:44:25.513330 27019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260928 06:44:25.515753 27034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:25.517012 27019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:44:25.517120 27019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "35215848daac4ff3865433cf1c666fb0"
format_stamp: "Formatted at 2026-09-28 06:44:25 on dist-test-slave-gtr8"
server_key: "85a5d8ee9229167443e671906c8e67ec"
server_key_iv: "950644d52fe50dab79bb2be83c4e2504"
server_key_version: "encryptionkey@0"
I20260928 06:44:25.517212 27019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:25.547340 27019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:25.550459 27019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:25.550645 27019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:25.551257 27019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:25.552112 27019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:25.552161 27019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:25.552214 27019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:25.552242 27019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:25.561133 27019 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:37933
I20260928 06:44:25.561190 27148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:37933 every 8 connection(s)
I20260928 06:44:25.562045 27019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:44:25.571956 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 27019
I20260928 06:44:25.572109 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:44:25.572433 8799 external_mini_cluster.cc:1428] Setting key af8ff2c4b8033c5e69cc5bba46a44dc6
Sep 28 06:44:25 dist-test-slave-gtr8 krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:25.578167 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:25.563975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:35161 (local address 127.8.151.254:41559)
0928 06:44:25.564113 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:25.564118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:25.564979 (+ 861us) server_negotiation.cc:407] Connection header received
0928 06:44:25.566616 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:25.566619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:25.566665 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:25.566741 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:25.568193 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:25.568759 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:25.570286 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:25.570534 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:25.570647 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:25.573393 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:25.573397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:25.573399 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:25.573423 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:25.576189 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:25.576687 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:25.576691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:25.576693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:25.576773 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:25.577104 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:25.577107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:25.577109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:25.577291 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:25.577373 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:25.577878 (+ 505us) server_negotiation.cc:299] Negotiation successful
0928 06:44:25.578003 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:44:25.579216 27151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:25.564339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41559 (local address 127.8.151.193:35161)
0928 06:44:25.564819 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:25.564873 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:25.566355 (+ 1482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:25.566926 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:25.566935 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:25.567418 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:25.568030 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:25.568048 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:25.568876 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:25.568879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:25.570145 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:25.570156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:25.570670 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:25.570673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:25.570852 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:25.571458 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:25.571477 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:25.573245 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:25.576306 (+ 3061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:25.576313 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:25.576328 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:25.576591 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:25.576856 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:25.576859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:25.576861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:25.577004 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:25.577390 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:25.577396 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:25.577642 (+ 246us) client_negotiation.cc:769] Sending connection context
0928 06:44:25.577820 (+ 178us) client_negotiation.cc:240] Negotiation successful
0928 06:44:25.578170 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":104,"threads_started":1}
I20260928 06:44:25.580457 27149 heartbeater.cc:347] Connected to a master server at 127.8.151.254:41559
I20260928 06:44:25.580694 27149 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:25.581153 27149 heartbeater.cc:511] Master 127.8.151.254:41559 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:44:25.582569 26957 ts_manager.cc:194] Registered new tserver with Master: 35215848daac4ff3865433cf1c666fb0 (127.8.151.193:37933)
I20260928 06:44:25.583720 26957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:35161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:44:25.601867 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41559
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:25.734803 27154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:25.735060 27154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:25.735103 27154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:25.739130 27154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:25.739217 27154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:25.739300 27154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:44:25.744674 27154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:25.745844 27154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:25.746845 27154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:25.755961 27159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:25.755971 27162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:25.755971 27160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:25.756163 27154 server_base.cc:1034] running on GCE node
I20260928 06:44:25.756660 27154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:25.757259 27154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:25.758422 27154 hybrid_clock.cc:648] HybridClock initialized: now 1790577865758400 us; error 33 us; skew 500 ppm
Sep 28 06:44:25 dist-test-slave-gtr8 krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:25.761152 27154 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:44:25.761981 27154 webserver.cc:460] Webserver started at http://127.8.151.194:41397/ using document root <none> and password file <none>
I20260928 06:44:25.762508 27154 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:25.762571 27154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:25.762746 27154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:25.764813 27154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "82048b9b43f34a6684119904640e3bf7"
format_stamp: "Formatted at 2026-09-28 06:44:25 on dist-test-slave-gtr8"
server_key: "206e00e107356612de56265cec50c684"
server_key_iv: "5421433e733a36344f9441cebf22a60e"
server_key_version: "encryptionkey@0"
I20260928 06:44:25.765364 27154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "82048b9b43f34a6684119904640e3bf7"
format_stamp: "Formatted at 2026-09-28 06:44:25 on dist-test-slave-gtr8"
server_key: "206e00e107356612de56265cec50c684"
server_key_iv: "5421433e733a36344f9441cebf22a60e"
server_key_version: "encryptionkey@0"
I20260928 06:44:25.769142 27154 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:44:25.771623 27169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:25.772876 27154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:44:25.772975 27154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "82048b9b43f34a6684119904640e3bf7"
format_stamp: "Formatted at 2026-09-28 06:44:25 on dist-test-slave-gtr8"
server_key: "206e00e107356612de56265cec50c684"
server_key_iv: "5421433e733a36344f9441cebf22a60e"
server_key_version: "encryptionkey@0"
I20260928 06:44:25.773095 27154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:25.787317 27154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:25.790326 27154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:25.790498 27154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:25.791102 27154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:25.791939 27154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:25.791985 27154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:25.792025 27154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:25.792054 27154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:25.800314 27154 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:42445
I20260928 06:44:25.800369 27283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:42445 every 8 connection(s)
I20260928 06:44:25.801210 27154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:44:25.808665 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 27154
I20260928 06:44:25.808816 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:44:25.809085 8799 external_mini_cluster.cc:1428] Setting key 0a442acb2d1f4c38f47c0c76c67aecae
Sep 28 06:44:25 dist-test-slave-gtr8 krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:25.816273 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:25.802966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:50345 (local address 127.8.151.254:41559)
0928 06:44:25.803162 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:25.803168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:25.804175 (+ 1007us) server_negotiation.cc:407] Connection header received
0928 06:44:25.805841 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:25.805844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:25.805893 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:25.805973 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:25.807548 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:25.808105 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:25.809697 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:25.809936 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:25.810029 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:25.812826 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:25.812830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:25.812852 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:25.812883 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:25.814178 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:25.814658 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:25.814661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:25.814663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:25.814746 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:25.815097 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:25.815100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:25.815102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:25.815325 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:25.815390 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:25.815915 (+ 525us) server_negotiation.cc:299] Negotiation successful
0928 06:44:25.816031 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 06:44:25.817224 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:25.803447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41559 (local address 127.8.151.194:50345)
0928 06:44:25.803977 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:25.804040 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:25.805582 (+ 1542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:25.806184 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:25.806193 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:25.806705 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:25.807399 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:25.807417 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:25.808223 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:25.808226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:25.809582 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:25.809593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:25.810087 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:25.810091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:25.810279 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:25.810905 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:25.810926 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:25.812664 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:25.814290 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:25.814296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:25.814311 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:25.814555 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:25.814845 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:25.814848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:25.814850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:25.814982 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:25.815447 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:25.815451 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:25.815675 (+ 224us) client_negotiation.cc:769] Sending connection context
0928 06:44:25.815837 (+ 162us) client_negotiation.cc:240] Negotiation successful
0928 06:44:25.816164 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":109,"threads_started":1}
I20260928 06:44:25.818320 27284 heartbeater.cc:347] Connected to a master server at 127.8.151.254:41559
I20260928 06:44:25.818547 27284 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:25.819010 27284 heartbeater.cc:511] Master 127.8.151.254:41559 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
I20260928 06:44:25.819999 26957 ts_manager.cc:194] Registered new tserver with Master: 82048b9b43f34a6684119904640e3bf7 (127.8.151.194:42445)
I20260928 06:44:25.820647 26957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:50345
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:44:25.838964 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41559
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:25.986409 27289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:25.986775 27289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:25.986833 27289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:25.991763 27289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:25.991829 27289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:25.991892 27289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:44:25.997193 27289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:41559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:25.998237 27289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:25.999256 27289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:26.008512 27294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:26.008522 27297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:26.008509 27295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:26.008839 27289 server_base.cc:1034] running on GCE node
I20260928 06:44:26.009196 27289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:26.009807 27289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:26.010979 27289 hybrid_clock.cc:648] HybridClock initialized: now 1790577866010956 us; error 32 us; skew 500 ppm
Sep 28 06:44:26 dist-test-slave-gtr8 krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577866, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:26.013657 27289 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:44:26.014505 27289 webserver.cc:460] Webserver started at http://127.8.151.195:45241/ using document root <none> and password file <none>
I20260928 06:44:26.015023 27289 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:26.015087 27289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:26.015468 27289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:26.017412 27289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "ea9bd5cdb7034e479ff131a96cc47251"
format_stamp: "Formatted at 2026-09-28 06:44:26 on dist-test-slave-gtr8"
server_key: "00bde408ab8239149c231ad08775d653"
server_key_iv: "efa7580f136e20a1db844ac33daa36c0"
server_key_version: "encryptionkey@0"
I20260928 06:44:26.017931 27289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "ea9bd5cdb7034e479ff131a96cc47251"
format_stamp: "Formatted at 2026-09-28 06:44:26 on dist-test-slave-gtr8"
server_key: "00bde408ab8239149c231ad08775d653"
server_key_iv: "efa7580f136e20a1db844ac33daa36c0"
server_key_version: "encryptionkey@0"
I20260928 06:44:26.021806 27289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:44:26.024276 27304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:26.025467 27289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:44:26.025568 27289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "ea9bd5cdb7034e479ff131a96cc47251"
format_stamp: "Formatted at 2026-09-28 06:44:26 on dist-test-slave-gtr8"
server_key: "00bde408ab8239149c231ad08775d653"
server_key_iv: "efa7580f136e20a1db844ac33daa36c0"
server_key_version: "encryptionkey@0"
I20260928 06:44:26.025715 27289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:26.065351 27289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:26.068485 27289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:26.068663 27289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:26.069252 27289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:26.070102 27289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:26.070150 27289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:26.070200 27289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:26.070235 27289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:26.079406 27289 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:35901
I20260928 06:44:26.079463 27418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:35901 every 8 connection(s)
I20260928 06:44:26.080282 27289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:44:26.086593 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 27289
I20260928 06:44:26.086716 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:44:26.087021 8799 external_mini_cluster.cc:1428] Setting key 2a97ce2281a8133eb60930faad5ffc79
Sep 28 06:44:26 dist-test-slave-gtr8 krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577866, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:26.094292 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.081993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:58029 (local address 127.8.151.254:41559)
0928 06:44:26.082137 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:26.082142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:26.083046 (+ 904us) server_negotiation.cc:407] Connection header received
0928 06:44:26.084549 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:26.084552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:26.084597 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:26.084668 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:26.086056 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.086587 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.087987 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.088256 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.088331 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.090998 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:26.091002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:26.091005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:26.091030 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:26.092412 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:26.092881 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:26.092884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:26.092886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:26.092962 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:26.093267 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:26.093270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:26.093272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:26.093451 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:26.093534 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:26.094017 (+ 483us) server_negotiation.cc:299] Negotiation successful
0928 06:44:26.094133 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:44:26.095078 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.082394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:41559 (local address 127.8.151.195:58029)
0928 06:44:26.082870 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:26.082929 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:26.084333 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:26.084855 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:26.084863 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:26.085317 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:26.085925 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.085943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.086719 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.086722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.087851 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.087862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.088529 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.088533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.088725 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.089323 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:26.089339 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:26.090849 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:26.092521 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:26.092527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:26.092542 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:26.092788 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:26.093043 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:26.093046 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:26.093048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:26.093181 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:26.093546 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:26.093553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:26.093795 (+ 242us) client_negotiation.cc:769] Sending connection context
0928 06:44:26.093961 (+ 166us) client_negotiation.cc:240] Negotiation successful
0928 06:44:26.094287 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260928 06:44:26.096160 27419 heartbeater.cc:347] Connected to a master server at 127.8.151.254:41559
I20260928 06:44:26.096401 27419 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:26.096935 27419 heartbeater.cc:511] Master 127.8.151.254:41559 requested a full tablet report, sending...
I20260928 06:44:26.097954 26957 ts_manager.cc:194] Registered new tserver with Master: ea9bd5cdb7034e479ff131a96cc47251 (127.8.151.195:35901)
I20260928 06:44:26.098487 26957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:58029
I20260928 06:44:26.100821 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:44:26.110441 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.102732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47778 (local address 127.8.151.254:41559)
0928 06:44:26.102849 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:26.102854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:26.103030 (+ 176us) server_negotiation.cc:407] Connection header received
0928 06:44:26.103124 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:26.103127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:26.103205 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:26.103282 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:26.103896 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.104629 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.105450 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.105746 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.105842 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.107285 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:26.107288 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:26.107291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:26.107316 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:26.108653 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:26.109122 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:26.109125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:26.109127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:26.109212 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:26.109569 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:26.109572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:26.109574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:26.109817 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:26.109953 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:26.110206 (+ 253us) server_negotiation.cc:299] Negotiation successful
0928 06:44:26.110296 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 06:44:26.113420 26957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:44:26.117229 26957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:44:26.132943 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.126525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42445 (local address 127.0.0.1:39042)
0928 06:44:26.126881 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:26.126933 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:26.127057 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:26.127751 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:26.127758 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:26.127780 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:26.127927 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.127934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.129802 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.129805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.131242 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.131253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.132423 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.132427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.132536 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.132576 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 06:44:26.132661 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 06:44:26.132793 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":13,"thread_start_us":96,"threads_started":1}
I20260928 06:44:26.132993 27433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.127392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.8.151.193:37933)
0928 06:44:26.127647 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:26.127653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:26.127882 (+ 229us) server_negotiation.cc:407] Connection header received
0928 06:44:26.128081 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:26.128086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:26.128283 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:26.128424 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:26.128642 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.129655 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.131381 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.132186 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.132293 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.132446 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:26.132779 (+ 333us) server_negotiation.cc:299] Negotiation successful
0928 06:44:26.132875 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
I20260928 06:44:26.133754 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.126664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39042 (local address 127.8.151.194:42445)
0928 06:44:26.127244 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:26.127258 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:26.127281 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:44:26.127359 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:26.127365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:26.127578 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:26.127772 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:26.128571 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.129630 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.131545 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.132302 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.133327 (+ 1025us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.133472 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:26.133523 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 06:44:26.133614 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":128,"threads_started":1}
I20260928 06:44:26.133948 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.126859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:37933 (local address 127.0.0.1:50062)
0928 06:44:26.127785 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:26.127804 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:26.127943 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:26.128404 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:26.128407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:26.128423 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:26.128498 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.128504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.129790 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.129793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.131242 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.131253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.132499 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.132502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.132624 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.132642 (+ 18us) client_negotiation.cc:769] Sending connection context
0928 06:44:26.132697 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 06:44:26.133674 (+ 977us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":1033,"thread_start_us":97,"threads_started":1}
I20260928 06:44:26.134168 27432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.127567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:35901 (local address 127.0.0.1:35572)
0928 06:44:26.128571 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:26.128584 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:26.128833 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:26.129294 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:26.129297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:26.129315 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:26.129601 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.129609 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.131105 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.131108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.132639 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.132650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.133807 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.133810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.133891 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.133912 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 06:44:26.133973 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 06:44:26.134032 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"mutex_wait_us":17,"thread_start_us":99,"threads_started":1}
I20260928 06:44:26.135377 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.127990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35572 (local address 127.8.151.195:35901)
0928 06:44:26.128356 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:26.128366 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:26.128763 (+ 397us) server_negotiation.cc:407] Connection header received
0928 06:44:26.128978 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:26.128983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:26.129159 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:26.129298 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:26.129725 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.130720 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.132769 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.133529 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.133627 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.133749 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:26.134375 (+ 626us) server_negotiation.cc:299] Negotiation successful
0928 06:44:26.135244 (+ 869us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":166,"threads_started":1}
I20260928 06:44:26.136976 27083 tablet_service.cc:1467] Processing CreateTablet for tablet f08e42b69d3241de8c2387a5567f2f6a (DEFAULT_TABLE table=test-table [id=f2dec8ec8e6d41319082490997af91b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:26.136853 27217 tablet_service.cc:1467] Processing CreateTablet for tablet f08e42b69d3241de8c2387a5567f2f6a (DEFAULT_TABLE table=test-table [id=f2dec8ec8e6d41319082490997af91b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:26.137991 27083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08e42b69d3241de8c2387a5567f2f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:26.137991 27217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08e42b69d3241de8c2387a5567f2f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:26.137888 27352 tablet_service.cc:1467] Processing CreateTablet for tablet f08e42b69d3241de8c2387a5567f2f6a (DEFAULT_TABLE table=test-table [id=f2dec8ec8e6d41319082490997af91b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:26.138895 27352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f08e42b69d3241de8c2387a5567f2f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:26.146209 27436 tablet_bootstrap.cc:492] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0: Bootstrap starting.
I20260928 06:44:26.147472 27437 tablet_bootstrap.cc:492] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7: Bootstrap starting.
I20260928 06:44:26.148347 27436 tablet_bootstrap.cc:654] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:26.149259 27436 log.cc:824] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:26.149737 27437 tablet_bootstrap.cc:654] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:26.149974 27438 tablet_bootstrap.cc:492] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: Bootstrap starting.
I20260928 06:44:26.150903 27437 log.cc:824] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:26.151944 27438 tablet_bootstrap.cc:654] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:26.153050 27437 tablet_bootstrap.cc:492] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7: No bootstrap required, opened a new log
I20260928 06:44:26.153033 27438 log.cc:824] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:26.153234 27437 ts_tablet_manager.cc:1397] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260928 06:44:26.154923 27436 tablet_bootstrap.cc:492] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0: No bootstrap required, opened a new log
I20260928 06:44:26.154961 27438 tablet_bootstrap.cc:492] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: No bootstrap required, opened a new log
I20260928 06:44:26.155160 27438 ts_tablet_manager.cc:1397] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:44:26.155257 27436 ts_tablet_manager.cc:1397] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260928 06:44:26.157027 27437 raft_consensus.cc:348] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 } } peers { permanent_uuid: "ea9bd5cdb7034e479ff131a96cc47251" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35901 } } peers { permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 } }
I20260928 06:44:26.157302 27437 raft_consensus.cc:374] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:26.157356 27437 raft_consensus.cc:729] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82048b9b43f34a6684119904640e3bf7, State: Initialized, Role: FOLLOWER
I20260928 06:44:26.157826 27437 consensus_queue.cc:260] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 } } peers { permanent_uuid: "ea9bd5cdb7034e479ff131a96cc47251" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35901 } } peers { permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 } }
I20260928 06:44:26.158829 27437 ts_tablet_manager.cc:1428] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260928 06:44:26.158782 27438 raft_consensus.cc:348] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 } } peers { permanent_uuid: "ea9bd5cdb7034e479ff131a96cc47251" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35901 } } peers { permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 } }
I20260928 06:44:26.159053 27438 raft_consensus.cc:374] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:26.159109 27438 raft_consensus.cc:729] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea9bd5cdb7034e479ff131a96cc47251, State: Initialized, Role: FOLLOWER
I20260928 06:44:26.159205 27284 heartbeater.cc:503] Master 127.8.151.254:41559 was elected leader, sending a full tablet report...
I20260928 06:44:26.159559 27438 consensus_queue.cc:260] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 } } peers { permanent_uuid: "ea9bd5cdb7034e479ff131a96cc47251" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35901 } } peers { permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 } }
I20260928 06:44:26.160120 27419 heartbeater.cc:503] Master 127.8.151.254:41559 was elected leader, sending a full tablet report...
I20260928 06:44:26.160475 27438 ts_tablet_manager.cc:1428] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260928 06:44:26.160691 27436 raft_consensus.cc:348] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 } } peers { permanent_uuid: "ea9bd5cdb7034e479ff131a96cc47251" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35901 } } peers { permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 } }
I20260928 06:44:26.160918 27436 raft_consensus.cc:374] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:26.160964 27436 raft_consensus.cc:729] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35215848daac4ff3865433cf1c666fb0, State: Initialized, Role: FOLLOWER
I20260928 06:44:26.161396 27436 consensus_queue.cc:260] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 } } peers { permanent_uuid: "ea9bd5cdb7034e479ff131a96cc47251" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35901 } } peers { permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 } }
I20260928 06:44:26.162289 27436 ts_tablet_manager.cc:1428] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260928 06:44:26.163103 27149 heartbeater.cc:503] Master 127.8.151.254:41559 was elected leader, sending a full tablet report...
I20260928 06:44:26.262166 27443 raft_consensus.cc:482] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:44:26.262410 27443 raft_consensus.cc:504] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 } } peers { permanent_uuid: "ea9bd5cdb7034e479ff131a96cc47251" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35901 } } peers { permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 } }
I20260928 06:44:26.263847 27443 leader_election.cc:288] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82048b9b43f34a6684119904640e3bf7 (127.8.151.194:42445), 35215848daac4ff3865433cf1c666fb0 (127.8.151.193:37933)
I20260928 06:44:26.268507 27433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.264241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:41623 (local address 127.8.151.193:37933)
0928 06:44:26.264365 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:26.264369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:26.264729 (+ 360us) server_negotiation.cc:407] Connection header received
0928 06:44:26.264896 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:26.264898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:26.264943 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:26.265004 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:26.265289 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.266088 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.267364 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.267894 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.268011 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.268074 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:26.268288 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 06:44:26.268369 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 06:44:26.268816 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.264100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42445 (local address 127.8.151.195:44171)
0928 06:44:26.264495 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:26.264511 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:26.264630 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:26.265006 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:26.265009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:26.265028 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:26.265171 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.265177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.266213 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.266217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.267529 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.267540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.268419 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.268422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.268523 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.268544 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 06:44:26.268622 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 06:44:26.268694 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":82,"spinlock_wait_cycles":27776}
I20260928 06:44:26.268842 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.264099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:37933 (local address 127.8.151.195:41623)
0928 06:44:26.264644 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:26.264656 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:26.264789 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:26.265028 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:26.265031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:26.265049 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:26.265179 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.265184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.266244 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.266247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.267248 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.267259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.268012 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.268015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.268117 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.268152 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 06:44:26.268275 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 06:44:26.268382 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":145,"thread_start_us":117,"threads_started":1}
I20260928 06:44:26.269351 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.264354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:44171 (local address 127.8.151.194:42445)
0928 06:44:26.264468 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:26.264472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:26.264592 (+ 120us) server_negotiation.cc:407] Connection header received
0928 06:44:26.264741 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:26.264744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:26.264794 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:26.264890 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:26.265321 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.266099 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.267660 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.268208 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.268314 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.268359 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:26.269135 (+ 776us) server_negotiation.cc:299] Negotiation successful
0928 06:44:26.269238 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 06:44:26.269486 27103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08e42b69d3241de8c2387a5567f2f6a" candidate_uuid: "ea9bd5cdb7034e479ff131a96cc47251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35215848daac4ff3865433cf1c666fb0" is_pre_election: true
I20260928 06:44:26.269829 27103 raft_consensus.cc:2454] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea9bd5cdb7034e479ff131a96cc47251 in term 0.
I20260928 06:44:26.270028 27237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08e42b69d3241de8c2387a5567f2f6a" candidate_uuid: "ea9bd5cdb7034e479ff131a96cc47251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82048b9b43f34a6684119904640e3bf7" is_pre_election: true
I20260928 06:44:26.270354 27237 raft_consensus.cc:2454] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea9bd5cdb7034e479ff131a96cc47251 in term 0.
I20260928 06:44:26.270339 27306 leader_election.cc:302] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35215848daac4ff3865433cf1c666fb0, ea9bd5cdb7034e479ff131a96cc47251; no voters:
I20260928 06:44:26.270673 27443 raft_consensus.cc:2781] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:44:26.270753 27443 raft_consensus.cc:482] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:44:26.270789 27443 raft_consensus.cc:3037] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:26.271862 27443 raft_consensus.cc:504] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 } } peers { permanent_uuid: "ea9bd5cdb7034e479ff131a96cc47251" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35901 } } peers { permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 } }
I20260928 06:44:26.272182 27443 leader_election.cc:288] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [CANDIDATE]: Term 1 election: Requested vote from peers 82048b9b43f34a6684119904640e3bf7 (127.8.151.194:42445), 35215848daac4ff3865433cf1c666fb0 (127.8.151.193:37933)
I20260928 06:44:26.272512 27237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08e42b69d3241de8c2387a5567f2f6a" candidate_uuid: "ea9bd5cdb7034e479ff131a96cc47251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82048b9b43f34a6684119904640e3bf7"
I20260928 06:44:26.272516 27103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f08e42b69d3241de8c2387a5567f2f6a" candidate_uuid: "ea9bd5cdb7034e479ff131a96cc47251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35215848daac4ff3865433cf1c666fb0"
I20260928 06:44:26.272651 27237 raft_consensus.cc:3037] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:26.272650 27103 raft_consensus.cc:3037] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:26.273622 27103 raft_consensus.cc:2454] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea9bd5cdb7034e479ff131a96cc47251 in term 1.
I20260928 06:44:26.273907 27306 leader_election.cc:302] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35215848daac4ff3865433cf1c666fb0, ea9bd5cdb7034e479ff131a96cc47251; no voters:
I20260928 06:44:26.273912 27237 raft_consensus.cc:2454] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea9bd5cdb7034e479ff131a96cc47251 in term 1.
I20260928 06:44:26.274073 27443 raft_consensus.cc:2781] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:26.274297 27443 raft_consensus.cc:686] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 1 LEADER]: Becoming Leader. State: Replica: ea9bd5cdb7034e479ff131a96cc47251, State: Running, Role: LEADER
I20260928 06:44:26.274668 27443 consensus_queue.cc:237] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 } } peers { permanent_uuid: "ea9bd5cdb7034e479ff131a96cc47251" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35901 } } peers { permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 } }
I20260928 06:44:26.278118 26957 catalog_manager.cc:5495] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea9bd5cdb7034e479ff131a96cc47251 (127.8.151.195). New cstate: current_term: 1 leader_uuid: "ea9bd5cdb7034e479ff131a96cc47251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea9bd5cdb7034e479ff131a96cc47251" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 35901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 } health_report { overall_health: UNKNOWN } } }
W20260928 06:44:26.302469 27285 tablet.cc:2369] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 06:44:26.313499 27150 tablet.cc:2369] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:44:26.324306 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.320304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35580 (local address 127.8.151.195:35901)
0928 06:44:26.320423 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:26.320427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:26.320441 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:44:26.320522 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:26.320526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:26.320591 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:26.320693 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:26.320932 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.321562 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.322450 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.322813 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.322916 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.323290 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:26.323832 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:26.323975 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:26.324124 (+ 149us) server_negotiation.cc:299] Negotiation successful
0928 06:44:26.324167 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 06:44:26.330349 27103 raft_consensus.cc:1261] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0 [term 1 FOLLOWER]: Refusing update from remote peer ea9bd5cdb7034e479ff131a96cc47251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:26.330367 27237 raft_consensus.cc:1261] T f08e42b69d3241de8c2387a5567f2f6a P 82048b9b43f34a6684119904640e3bf7 [term 1 FOLLOWER]: Refusing update from remote peer ea9bd5cdb7034e479ff131a96cc47251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260928 06:44:26.331187 27420 tablet.cc:2369] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:44:26.331100 27446 consensus_queue.cc:1035] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35215848daac4ff3865433cf1c666fb0" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 37933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:26.331573 27446 consensus_queue.cc:1035] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82048b9b43f34a6684119904640e3bf7" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:26.338382 27449 mvcc.cc:205] Tried to move back new op lower bound from 7334206940481302528 to 7334206940263854080. Current Snapshot: MvccSnapshot[applied={T|T < 7334206940481302528}]
I20260928 06:44:26.339840 27451 mvcc.cc:205] Tried to move back new op lower bound from 7334206940481302528 to 7334206940263854080. Current Snapshot: MvccSnapshot[applied={T|T < 7334206940481302528}]
I20260928 06:44:26.347550 26957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:26.347700 26957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:26.349804 26957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd64bbbe6ed34e01aeecc834a2ec3ca9: Sending DeleteTablet for 3 replicas of tablet f08e42b69d3241de8c2387a5567f2f6a
I20260928 06:44:26.350571 27217 tablet_service.cc:1514] Processing DeleteTablet for tablet f08e42b69d3241de8c2387a5567f2f6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:26 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:39042
I20260928 06:44:26.350657 27352 tablet_service.cc:1514] Processing DeleteTablet for tablet f08e42b69d3241de8c2387a5567f2f6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:26 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:35572
I20260928 06:44:26.350760 27083 tablet_service.cc:1514] Processing DeleteTablet for tablet f08e42b69d3241de8c2387a5567f2f6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:26 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:50062
I20260928 06:44:26.351099 27459 tablet_replica.cc:330] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: stopping tablet replica
I20260928 06:44:26.351164 27460 tablet_replica.cc:330] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0: stopping tablet replica
I20260928 06:44:26.351430 27459 raft_consensus.cc:2229] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:44:26.351433 27460 raft_consensus.cc:2229] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:44:26.351743 27460 raft_consensus.cc:2258] T f08e42b69d3241de8c2387a5567f2f6a P 35215848daac4ff3865433cf1c666fb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:26.351771 27459 raft_consensus.cc:2258] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:26.351871 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 27019
I20260928 06:44:26.353933 27459 ts_tablet_manager.cc:1905] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:44:26.356598 27459 ts_tablet_manager.cc:1918] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:44:26.356693 27459 log.cc:1196] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/f08e42b69d3241de8c2387a5567f2f6a
I20260928 06:44:26.356995 27459 ts_tablet_manager.cc:1939] T f08e42b69d3241de8c2387a5567f2f6a P ea9bd5cdb7034e479ff131a96cc47251: Deleting consensus metadata
I20260928 06:44:26.358083 26943 catalog_manager.cc:4841] TS ea9bd5cdb7034e479ff131a96cc47251 (127.8.151.195:35901): tablet f08e42b69d3241de8c2387a5567f2f6a (table test-table [id=f2dec8ec8e6d41319082490997af91b4]) successfully deleted
W20260928 06:44:26.361038 26942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:37933 (error 108)
I20260928 06:44:26.361259 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 27154
I20260928 06:44:26.362865 27432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.362109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:37933 (local address 127.0.0.1:50066)
0928 06:44:26.362737 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:26.362797 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553}
W20260928 06:44:26.363003 26942 catalog_manager.cc:4584] TS 35215848daac4ff3865433cf1c666fb0 (127.8.151.193:37933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f08e42b69d3241de8c2387a5567f2f6a: Network error: Client connection negotiation failed: client connection to 127.8.151.193:37933: connect: Connection refused (error 111)
I20260928 06:44:26.367544 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 27289
I20260928 06:44:26.367676 27432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.367405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42445 (local address 127.0.0.1:39044)
0928 06:44:26.367549 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:26.367610 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.194:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260928 06:44:26.372427 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 26925
2026-09-28T06:44:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2467 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-28T06:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:26Z Disabled control of system clock
I20260928 06:44:26.402626 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:42511
--webserver_interface=127.8.151.254
--webserver_port=0
--builtin_ntp_servers=127.8.151.212:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:42511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 06:44:26.527786 27465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:26.528056 27465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:26.528092 27465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:26.532092 27465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:26.532148 27465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:26.532162 27465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:26.532177 27465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:26.532191 27465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:26.536880 27465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:42511
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:42511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:26.537932 27465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:26.539325 27465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:26.548619 27470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:26.548568 27471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:26.548918 27465 server_base.cc:1034] running on GCE node
W20260928 06:44:26.548585 27473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:26.549814 27465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:26.551394 27465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:26.552598 27465 hybrid_clock.cc:648] HybridClock initialized: now 1790577866552573 us; error 34 us; skew 500 ppm
I20260928 06:44:26.554816 27465 webserver.cc:460] Webserver started at http://127.8.151.254:45799/ using document root <none> and password file <none>
I20260928 06:44:26.555565 27465 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:26.555662 27465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:26.555912 27465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:26.558681 27465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "84c19f71eb25404794bac200257c2eb6"
format_stamp: "Formatted at 2026-09-28 06:44:26 on dist-test-slave-gtr8"
server_key: "7caacc4836e8c5bdd608e1b6063428f4"
server_key_iv: "12dcb25ff6a8f8636119ba3543a4eb4d"
server_key_version: "encryptionkey@0"
I20260928 06:44:26.559396 27465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "84c19f71eb25404794bac200257c2eb6"
format_stamp: "Formatted at 2026-09-28 06:44:26 on dist-test-slave-gtr8"
server_key: "7caacc4836e8c5bdd608e1b6063428f4"
server_key_iv: "12dcb25ff6a8f8636119ba3543a4eb4d"
server_key_version: "encryptionkey@0"
I20260928 06:44:26.564262 27465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260928 06:44:26.567432 27479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:26.569001 27465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260928 06:44:26.569146 27465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "84c19f71eb25404794bac200257c2eb6"
format_stamp: "Formatted at 2026-09-28 06:44:26 on dist-test-slave-gtr8"
server_key: "7caacc4836e8c5bdd608e1b6063428f4"
server_key_iv: "12dcb25ff6a8f8636119ba3543a4eb4d"
server_key_version: "encryptionkey@0"
I20260928 06:44:26.569278 27465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:26.585053 27465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:26.585870 27465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:26.586068 27465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:26.594503 27532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:42511 every 8 connection(s)
I20260928 06:44:26.594515 27465 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:42511
I20260928 06:44:26.595769 27465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:26.598631 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 27465
I20260928 06:44:26.598780 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:26.599015 27533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:26.599100 8799 external_mini_cluster.cc:1428] Setting key 5680e6621cc2ef97fc22cb9c2c1e02de
I20260928 06:44:26.605764 27533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6: Bootstrap starting.
I20260928 06:44:26.608296 27533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:26.609030 27533 log.cc:824] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:26.610496 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.600639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.8.151.254:42511)
0928 06:44:26.601413 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:26.601435 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:26.601481 (+ 46us) server_negotiation.cc:407] Connection header received
0928 06:44:26.602423 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:26.602446 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:26.602963 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:26.603352 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:26.603801 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.605210 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.606424 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.607104 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.607335 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.608220 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:26.608227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:26.608243 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:26.608746 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:26.608818 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:26.608846 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:26.608948 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:26.609110 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 06:44:26.609569 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":158,"threads_started":1}
I20260928 06:44:26.610966 27533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6: No bootstrap required, opened a new log
I20260928 06:44:26.614857 27533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c19f71eb25404794bac200257c2eb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42511 } }
I20260928 06:44:26.615074 27533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:26.615129 27533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c19f71eb25404794bac200257c2eb6, State: Initialized, Role: FOLLOWER
I20260928 06:44:26.615608 27533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c19f71eb25404794bac200257c2eb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42511 } }
I20260928 06:44:26.615767 27533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:26.615839 27533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:26.615935 27533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:26.616787 27533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c19f71eb25404794bac200257c2eb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42511 } }
I20260928 06:44:26.617100 27533 leader_election.cc:302] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84c19f71eb25404794bac200257c2eb6; no voters:
I20260928 06:44:26.617380 27533 leader_election.cc:288] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:26.617512 27538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:26.617748 27538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [term 1 LEADER]: Becoming Leader. State: Replica: 84c19f71eb25404794bac200257c2eb6, State: Running, Role: LEADER
I20260928 06:44:26.618052 27538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c19f71eb25404794bac200257c2eb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42511 } }
I20260928 06:44:26.618584 27533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:26.619261 27538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84c19f71eb25404794bac200257c2eb6. Latest consensus state: current_term: 1 leader_uuid: "84c19f71eb25404794bac200257c2eb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c19f71eb25404794bac200257c2eb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42511 } } }
I20260928 06:44:26.619387 27538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:26.620028 27540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84c19f71eb25404794bac200257c2eb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c19f71eb25404794bac200257c2eb6" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 42511 } } }
I20260928 06:44:26.620132 27540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:26.620528 27546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:26.624368 27546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:26.630591 27546 catalog_manager.cc:1347] Generated new cluster ID: c9d82ebab8fd42458ae35949202250e4
I20260928 06:44:26.630666 27546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:26.652127 27546 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:26.653322 27546 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:26.668033 27546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6: Generated new TSK 0
I20260928 06:44:26.668951 27546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:26.678907 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42511
--builtin_ntp_servers=127.8.151.212:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 06:44:26.822263 27557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:26.822540 27557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:26.822593 27557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:26.826630 27557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:26.826686 27557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:26.826759 27557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:44:26.831615 27557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:26.832722 27557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:26.834095 27557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:26.843741 27562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:26.843961 27557 server_base.cc:1034] running on GCE node
W20260928 06:44:26.843756 27563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:26.843845 27565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:26.844471 27557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:26.845124 27557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:26.846298 27557 hybrid_clock.cc:648] HybridClock initialized: now 1790577866846277 us; error 34 us; skew 500 ppm
I20260928 06:44:26.848057 27557 webserver.cc:460] Webserver started at http://127.8.151.193:38957/ using document root <none> and password file <none>
I20260928 06:44:26.848629 27557 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:26.848687 27557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:26.848869 27557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:26.850831 27557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "ddfc05d6f35143cab6a47186b9967b77"
format_stamp: "Formatted at 2026-09-28 06:44:26 on dist-test-slave-gtr8"
server_key: "399b237c428c6b06a9d2c61e0760a15c"
server_key_iv: "d58ffa77f3f115e6386b7d76ae54cb1d"
server_key_version: "encryptionkey@0"
I20260928 06:44:26.851387 27557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "ddfc05d6f35143cab6a47186b9967b77"
format_stamp: "Formatted at 2026-09-28 06:44:26 on dist-test-slave-gtr8"
server_key: "399b237c428c6b06a9d2c61e0760a15c"
server_key_iv: "d58ffa77f3f115e6386b7d76ae54cb1d"
server_key_version: "encryptionkey@0"
I20260928 06:44:26.854951 27557 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260928 06:44:26.857458 27571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:26.858697 27557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:44:26.858807 27557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "ddfc05d6f35143cab6a47186b9967b77"
format_stamp: "Formatted at 2026-09-28 06:44:26 on dist-test-slave-gtr8"
server_key: "399b237c428c6b06a9d2c61e0760a15c"
server_key_iv: "d58ffa77f3f115e6386b7d76ae54cb1d"
server_key_version: "encryptionkey@0"
I20260928 06:44:26.858901 27557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:26.872398 27557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:26.873045 27557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:26.873209 27557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:26.873773 27557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:26.874717 27557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:26.874766 27557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:26.874816 27557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:26.874842 27557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:26.884636 27557 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:38439
I20260928 06:44:26.884684 27685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:38439 every 8 connection(s)
I20260928 06:44:26.885757 27557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:44:26.887485 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 27557
I20260928 06:44:26.887598 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:44:26.887848 8799 external_mini_cluster.cc:1428] Setting key 13b1095668a6412c83f8ec342d4a8b76
I20260928 06:44:26.889689 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42511
--builtin_ntp_servers=127.8.151.212:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 06:44:26.895936 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.889131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:33821 (local address 127.8.151.254:42511)
0928 06:44:26.890971 (+ 1840us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:26.890976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:26.890990 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:44:26.891041 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:26.891044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:26.891099 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:26.891192 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:26.892374 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.893001 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.894149 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.894368 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.894455 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.895479 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:26.895483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:26.895485 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:26.895499 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:26.895517 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:26.895524 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:26.895582 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:26.895721 (+ 139us) server_negotiation.cc:299] Negotiation successful
0928 06:44:26.895793 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1737}
I20260928 06:44:26.896776 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:26.888165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42511 (local address 127.8.151.193:33821)
0928 06:44:26.888897 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:26.888948 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:26.890533 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:26.891509 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:26.891519 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:26.891636 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:26.892237 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.892253 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.893128 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.893130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.894031 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:26.894040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:26.894504 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:26.894508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:26.894664 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:26.895236 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:44:26.895237 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:26.895278 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:44:26.895338 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:26.895594 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:26.895603 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:26.895609 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:44:26.895712 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 06:44:26.895979 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":127,"threads_started":1}
I20260928 06:44:26.897935 27686 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42511
I20260928 06:44:26.898145 27686 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:26.898623 27686 heartbeater.cc:511] Master 127.8.151.254:42511 requested a full tablet report, sending...
I20260928 06:44:26.899996 27497 ts_manager.cc:194] Registered new tserver with Master: ddfc05d6f35143cab6a47186b9967b77 (127.8.151.193:38439)
I20260928 06:44:26.901104 27497 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.193:33821
W20260928 06:44:27.010322 27689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:27.010584 27689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:27.010622 27689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:27.014748 27689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:27.014815 27689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:27.014870 27689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:44:27.019795 27689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:27.020982 27689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:27.022387 27689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:27.031672 27694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:27.031649 27695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:27.031893 27697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:27.031875 27689 server_base.cc:1034] running on GCE node
I20260928 06:44:27.032392 27689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:27.033181 27689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:27.034381 27689 hybrid_clock.cc:648] HybridClock initialized: now 1790577867034350 us; error 45 us; skew 500 ppm
I20260928 06:44:27.036711 27689 webserver.cc:460] Webserver started at http://127.8.151.194:41481/ using document root <none> and password file <none>
I20260928 06:44:27.037426 27689 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:27.037523 27689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:27.037772 27689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:27.040297 27689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "9b2b7447d2f244c592e37ed1b3de6bc6"
format_stamp: "Formatted at 2026-09-28 06:44:27 on dist-test-slave-gtr8"
server_key: "e957f98011b039799e7083e66e8f815b"
server_key_iv: "1a6b5f1993f563efa492589fd821470a"
server_key_version: "encryptionkey@0"
I20260928 06:44:27.040830 27689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "9b2b7447d2f244c592e37ed1b3de6bc6"
format_stamp: "Formatted at 2026-09-28 06:44:27 on dist-test-slave-gtr8"
server_key: "e957f98011b039799e7083e66e8f815b"
server_key_iv: "1a6b5f1993f563efa492589fd821470a"
server_key_version: "encryptionkey@0"
I20260928 06:44:27.044677 27689 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 06:44:27.047252 27703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:27.048569 27689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:27.048682 27689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "9b2b7447d2f244c592e37ed1b3de6bc6"
format_stamp: "Formatted at 2026-09-28 06:44:27 on dist-test-slave-gtr8"
server_key: "e957f98011b039799e7083e66e8f815b"
server_key_iv: "1a6b5f1993f563efa492589fd821470a"
server_key_version: "encryptionkey@0"
I20260928 06:44:27.048801 27689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:27.060350 27689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:27.061033 27689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:27.061204 27689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:27.061789 27689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:27.062798 27689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:27.062847 27689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:27.062901 27689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:27.062927 27689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:27.071966 27689 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:36259
I20260928 06:44:27.072023 27816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:36259 every 8 connection(s)
I20260928 06:44:27.072857 27689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:44:27.076297 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 27689
I20260928 06:44:27.076411 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:44:27.076681 8799 external_mini_cluster.cc:1428] Setting key c37dd3aa3b9a1353b45aa9cc44a5ab71
I20260928 06:44:27.078678 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42511
--builtin_ntp_servers=127.8.151.212:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 06:44:27.083185 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.074720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:41323 (local address 127.8.151.254:42511)
0928 06:44:27.074874 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:27.074879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:27.076158 (+ 1279us) server_negotiation.cc:407] Connection header received
0928 06:44:27.077839 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:27.077843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:27.077894 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:27.077952 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:27.078999 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.079749 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.081365 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.081578 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.081669 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.082705 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:27.082708 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:27.082711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:27.082725 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:27.082752 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:27.082760 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:27.082817 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:27.082954 (+ 137us) server_negotiation.cc:299] Negotiation successful
0928 06:44:27.083015 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:44:27.084154 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.075247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42511 (local address 127.8.151.194:41323)
0928 06:44:27.075982 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:27.076034 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:27.077624 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:27.078126 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:27.078135 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:27.078240 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:27.078870 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.078885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.079873 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.079877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.081246 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.081257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.081728 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.081732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.081903 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.082455 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:44:27.082456 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:27.082497 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:44:27.082560 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:27.082828 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:27.082837 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:27.082842 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 06:44:27.082946 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 06:44:27.083339 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":107,"threads_started":1}
I20260928 06:44:27.085170 27817 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42511
I20260928 06:44:27.085395 27817 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:27.085876 27817 heartbeater.cc:511] Master 127.8.151.254:42511 requested a full tablet report, sending...
I20260928 06:44:27.086813 27497 ts_manager.cc:194] Registered new tserver with Master: 9b2b7447d2f244c592e37ed1b3de6bc6 (127.8.151.194:36259)
I20260928 06:44:27.087399 27497 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.194:41323
W20260928 06:44:27.222293 27820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:27.222626 27820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:27.222688 27820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:27.226637 27820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:27.226687 27820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:27.226761 27820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:44:27.231428 27820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:42511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:27.232448 27820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:27.233808 27820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:27.243091 27828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:27.243219 27825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:27.243083 27826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:27.243505 27820 server_base.cc:1034] running on GCE node
I20260928 06:44:27.243969 27820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:27.244756 27820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:27.245947 27820 hybrid_clock.cc:648] HybridClock initialized: now 1790577867245908 us; error 48 us; skew 500 ppm
I20260928 06:44:27.248221 27820 webserver.cc:460] Webserver started at http://127.8.151.195:42839/ using document root <none> and password file <none>
I20260928 06:44:27.248977 27820 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:27.249068 27820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:27.249329 27820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:27.252136 27820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "fa544452588348449bc45d91f6b9cb73"
format_stamp: "Formatted at 2026-09-28 06:44:27 on dist-test-slave-gtr8"
server_key: "00b1200f561756c0e35ee7756c9203d0"
server_key_iv: "9989b1c522367242405575b8c44fbb4f"
server_key_version: "encryptionkey@0"
I20260928 06:44:27.252921 27820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "fa544452588348449bc45d91f6b9cb73"
format_stamp: "Formatted at 2026-09-28 06:44:27 on dist-test-slave-gtr8"
server_key: "00b1200f561756c0e35ee7756c9203d0"
server_key_iv: "9989b1c522367242405575b8c44fbb4f"
server_key_version: "encryptionkey@0"
I20260928 06:44:27.257719 27820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 06:44:27.260867 27834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:27.262455 27820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 06:44:27.262575 27820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "fa544452588348449bc45d91f6b9cb73"
format_stamp: "Formatted at 2026-09-28 06:44:27 on dist-test-slave-gtr8"
server_key: "00b1200f561756c0e35ee7756c9203d0"
server_key_iv: "9989b1c522367242405575b8c44fbb4f"
server_key_version: "encryptionkey@0"
I20260928 06:44:27.262698 27820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:27.300159 27820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:27.300902 27820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:27.301074 27820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:27.301672 27820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:27.302666 27820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:27.302716 27820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:27.302776 27820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:27.302801 27820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:27.311371 27820 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:43259
I20260928 06:44:27.311436 27948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:43259 every 8 connection(s)
I20260928 06:44:27.312294 27820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:44:27.316202 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 27820
I20260928 06:44:27.316308 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:44:27.316546 8799 external_mini_cluster.cc:1428] Setting key 2a9b0a257c3d7ceac974cd5f46b829fa
I20260928 06:44:27.323380 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.314324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:49223 (local address 127.8.151.254:42511)
0928 06:44:27.314486 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:27.314492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:27.315718 (+ 1226us) server_negotiation.cc:407] Connection header received
0928 06:44:27.317702 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:27.317706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:27.317762 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:27.317842 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:27.318995 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.319606 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.321393 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.321617 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.321695 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.322909 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:27.322912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:27.322914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:27.322927 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:27.322944 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:27.322951 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:27.323004 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:27.323212 (+ 208us) server_negotiation.cc:299] Negotiation successful
0928 06:44:27.323265 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 06:44:27.324566 27951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.314704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:42511 (local address 127.8.151.195:49223)
0928 06:44:27.315542 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:27.315600 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:27.317449 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:27.318057 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:27.318067 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:27.318194 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:27.318866 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.318884 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.319730 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.319734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.321274 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.321286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.321770 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.321774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.321976 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.322614 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:44:27.322616 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:27.322672 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:44:27.322759 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:27.323026 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:27.323034 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:27.323040 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:44:27.323218 (+ 178us) client_negotiation.cc:240] Negotiation successful
0928 06:44:27.323522 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":114,"threads_started":1}
I20260928 06:44:27.325687 27949 heartbeater.cc:347] Connected to a master server at 127.8.151.254:42511
I20260928 06:44:27.325945 27949 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:27.326584 27949 heartbeater.cc:511] Master 127.8.151.254:42511 requested a full tablet report, sending...
I20260928 06:44:27.327735 27497 ts_manager.cc:194] Registered new tserver with Master: fa544452588348449bc45d91f6b9cb73 (127.8.151.195:43259)
I20260928 06:44:27.328454 27497 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.195:49223
I20260928 06:44:27.330291 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:44:27.337050 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.332626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.8.151.254:42511)
0928 06:44:27.332760 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:27.332765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:27.332897 (+ 132us) server_negotiation.cc:407] Connection header received
0928 06:44:27.332985 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:27.332987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:27.333024 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:27.333089 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:27.333419 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.333978 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.335214 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.335426 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.335511 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.336475 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:27.336478 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:27.336481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:27.336493 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:27.336513 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:27.336522 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:27.336640 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:27.336862 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 06:44:27.336907 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:44:27.340201 27497 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:44:27.344281 27497 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:44:27.360038 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.353067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:36259 (local address 127.0.0.1:35552)
0928 06:44:27.354430 (+ 1363us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:27.354440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:27.354507 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:27.355399 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:27.355401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:27.355416 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:27.355496 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.355502 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.356837 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.356840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.358169 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.358180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.359523 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.359527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.359631 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.359669 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 06:44:27.359775 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 06:44:27.359895 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1312,"thread_start_us":89,"threads_started":1}
I20260928 06:44:27.360513 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.352985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:43259 (local address 127.0.0.1:39838)
0928 06:44:27.353320 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:27.353373 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:27.353491 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:27.356327 (+ 2836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:27.356330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:27.356346 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:27.356434 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.356439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.357777 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.357780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.358941 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.358953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.360147 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.360152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.360234 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.360256 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 06:44:27.360332 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 06:44:27.360391 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":47,"thread_start_us":102,"threads_started":1}
I20260928 06:44:27.360948 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.353792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.8.151.195:43259)
0928 06:44:27.355796 (+ 2004us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:27.355806 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:27.355827 (+ 21us) server_negotiation.cc:407] Connection header received
0928 06:44:27.355895 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:27.355899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:27.356058 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:27.356189 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:27.356553 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.357643 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.359078 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.359835 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.359920 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.360083 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:27.360685 (+ 602us) server_negotiation.cc:299] Negotiation successful
0928 06:44:27.360797 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1897,"thread_start_us":146,"threads_started":1}
I20260928 06:44:27.361181 27963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.353771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35552 (local address 127.8.151.194:36259)
0928 06:44:27.354884 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:27.354892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:27.354910 (+ 18us) server_negotiation.cc:407] Connection header received
0928 06:44:27.354966 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:27.354969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:27.355107 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:27.355267 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:27.355606 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.356715 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.358317 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.359130 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.359257 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.359459 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:27.360806 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0928 06:44:27.360898 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":121,"threads_started":1}
I20260928 06:44:27.362241 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.353364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38439 (local address 127.0.0.1:42568)
0928 06:44:27.354620 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:27.354628 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:27.354690 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:27.355199 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:27.355205 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:27.355227 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:27.355380 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.355388 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.357561 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.357564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.358854 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.358865 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.361784 (+ 2919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.361792 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.361894 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.361918 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 06:44:27.361990 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 06:44:27.362052 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1217,"mutex_wait_us":31,"thread_start_us":102,"threads_started":1}
I20260928 06:44:27.363008 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.353906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42568 (local address 127.8.151.193:38439)
0928 06:44:27.354309 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:27.354320 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:27.354719 (+ 399us) server_negotiation.cc:407] Connection header received
0928 06:44:27.354809 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:27.354815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:27.355009 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:27.355193 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:27.355484 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.357430 (+ 1946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.358990 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.360730 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.360816 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.360977 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:27.362659 (+ 1682us) server_negotiation.cc:299] Negotiation successful
0928 06:44:27.362785 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":97,"threads_started":1}
I20260928 06:44:27.363989 27751 tablet_service.cc:1467] Processing CreateTablet for tablet 7e92af4cc82f44c589dc14bf20106b0a (DEFAULT_TABLE table=test-table [id=eed651f191d242068e126125c6eaae1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:27.363987 27882 tablet_service.cc:1467] Processing CreateTablet for tablet 7e92af4cc82f44c589dc14bf20106b0a (DEFAULT_TABLE table=test-table [id=eed651f191d242068e126125c6eaae1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:27.365077 27882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e92af4cc82f44c589dc14bf20106b0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:27.365077 27751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e92af4cc82f44c589dc14bf20106b0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:27.372370 27966 tablet_bootstrap.cc:492] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: Bootstrap starting.
I20260928 06:44:27.373772 27967 tablet_bootstrap.cc:492] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: Bootstrap starting.
I20260928 06:44:27.374151 27619 tablet_service.cc:1467] Processing CreateTablet for tablet 7e92af4cc82f44c589dc14bf20106b0a (DEFAULT_TABLE table=test-table [id=eed651f191d242068e126125c6eaae1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:27.374631 27966 tablet_bootstrap.cc:654] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:27.375485 27966 log.cc:824] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:27.376068 27619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e92af4cc82f44c589dc14bf20106b0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:27.376197 27967 tablet_bootstrap.cc:654] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:27.376950 27967 log.cc:824] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:27.378973 27967 tablet_bootstrap.cc:492] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: No bootstrap required, opened a new log
I20260928 06:44:27.379172 27967 ts_tablet_manager.cc:1397] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:44:27.380900 27966 tablet_bootstrap.cc:492] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: No bootstrap required, opened a new log
I20260928 06:44:27.381089 27966 ts_tablet_manager.cc:1397] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260928 06:44:27.382203 27970 tablet_bootstrap.cc:492] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77: Bootstrap starting.
I20260928 06:44:27.383343 27967 raft_consensus.cc:348] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 } } peers { permanent_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36259 } } peers { permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 } }
I20260928 06:44:27.383605 27967 raft_consensus.cc:374] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:27.383659 27967 raft_consensus.cc:729] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa544452588348449bc45d91f6b9cb73, State: Initialized, Role: FOLLOWER
I20260928 06:44:27.384090 27967 consensus_queue.cc:260] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 } } peers { permanent_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36259 } } peers { permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 } }
I20260928 06:44:27.384342 27970 tablet_bootstrap.cc:654] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:27.384761 27949 heartbeater.cc:503] Master 127.8.151.254:42511 was elected leader, sending a full tablet report...
I20260928 06:44:27.385113 27967 ts_tablet_manager.cc:1428] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:44:27.385128 27970 log.cc:824] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:27.385876 27966 raft_consensus.cc:348] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 } } peers { permanent_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36259 } } peers { permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 } }
I20260928 06:44:27.386135 27966 raft_consensus.cc:374] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:27.386181 27966 raft_consensus.cc:729] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b2b7447d2f244c592e37ed1b3de6bc6, State: Initialized, Role: FOLLOWER
I20260928 06:44:27.386683 27966 consensus_queue.cc:260] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 } } peers { permanent_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36259 } } peers { permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 } }
I20260928 06:44:27.386973 27970 tablet_bootstrap.cc:492] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77: No bootstrap required, opened a new log
I20260928 06:44:27.387157 27970 ts_tablet_manager.cc:1397] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:44:27.387359 27817 heartbeater.cc:503] Master 127.8.151.254:42511 was elected leader, sending a full tablet report...
I20260928 06:44:27.387660 27966 ts_tablet_manager.cc:1428] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260928 06:44:27.390923 27970 raft_consensus.cc:348] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 } } peers { permanent_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36259 } } peers { permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 } }
I20260928 06:44:27.391108 27970 raft_consensus.cc:374] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:27.391168 27970 raft_consensus.cc:729] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfc05d6f35143cab6a47186b9967b77, State: Initialized, Role: FOLLOWER
I20260928 06:44:27.391515 27970 consensus_queue.cc:260] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 } } peers { permanent_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36259 } } peers { permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 } }
I20260928 06:44:27.392082 27686 heartbeater.cc:503] Master 127.8.151.254:42511 was elected leader, sending a full tablet report...
I20260928 06:44:27.392350 27970 ts_tablet_manager.cc:1428] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:44:27.553422 27973 raft_consensus.cc:482] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:44:27.553607 27973 raft_consensus.cc:504] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 } } peers { permanent_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36259 } } peers { permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 } }
I20260928 06:44:27.554845 27973 leader_election.cc:288] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa544452588348449bc45d91f6b9cb73 (127.8.151.195:43259), ddfc05d6f35143cab6a47186b9967b77 (127.8.151.193:38439)
I20260928 06:44:27.559177 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.555175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:59111 (local address 127.8.151.195:43259)
0928 06:44:27.555305 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:27.555310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:27.555759 (+ 449us) server_negotiation.cc:407] Connection header received
0928 06:44:27.555975 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:27.555978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:27.556027 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:27.556105 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:27.556340 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.556909 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.557978 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.558570 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.558689 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.558740 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:27.558928 (+ 188us) server_negotiation.cc:299] Negotiation successful
0928 06:44:27.558999 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:44:27.559183 27975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.555070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:43259 (local address 127.8.151.194:59111)
0928 06:44:27.555662 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:27.555674 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:27.555840 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:27.556124 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:27.556126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:27.556140 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:27.556243 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.556249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.557007 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.557010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.557865 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.557874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.558690 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.558694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.558799 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.558827 (+ 28us) client_negotiation.cc:769] Sending connection context
0928 06:44:27.558904 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 06:44:27.558980 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":311,"thread_start_us":98,"threads_started":1}
I20260928 06:44:27.559423 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.555322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:54419 (local address 127.8.151.193:38439)
0928 06:44:27.555448 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:27.555452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:27.555466 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:44:27.555517 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:27.555520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:27.555578 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:27.555652 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:27.555883 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.556584 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.558088 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.558671 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.558766 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.558803 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:27.559239 (+ 436us) server_negotiation.cc:299] Negotiation successful
0928 06:44:27.559313 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 06:44:27.559655 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.555063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38439 (local address 127.8.151.194:54419)
0928 06:44:27.555193 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:27.555207 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:27.555310 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:27.555661 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:27.555664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:27.555682 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:27.555786 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.555791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.556715 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.556719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.557960 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:27.557973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.558854 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.558857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:27.558951 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.558972 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 06:44:27.559035 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 06:44:27.559100 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 06:44:27.559942 27902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e92af4cc82f44c589dc14bf20106b0a" candidate_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa544452588348449bc45d91f6b9cb73" is_pre_election: true
I20260928 06:44:27.560220 27640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e92af4cc82f44c589dc14bf20106b0a" candidate_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddfc05d6f35143cab6a47186b9967b77" is_pre_election: true
I20260928 06:44:27.560298 27902 raft_consensus.cc:2454] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b2b7447d2f244c592e37ed1b3de6bc6 in term 0.
I20260928 06:44:27.560490 27640 raft_consensus.cc:2454] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b2b7447d2f244c592e37ed1b3de6bc6 in term 0.
I20260928 06:44:27.560808 27704 leader_election.cc:302] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b2b7447d2f244c592e37ed1b3de6bc6, fa544452588348449bc45d91f6b9cb73; no voters:
I20260928 06:44:27.561095 27973 raft_consensus.cc:2781] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:44:27.561172 27973 raft_consensus.cc:482] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:44:27.561218 27973 raft_consensus.cc:3037] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:27.562258 27973 raft_consensus.cc:504] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 } } peers { permanent_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36259 } } peers { permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 } }
I20260928 06:44:27.562577 27973 leader_election.cc:288] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [CANDIDATE]: Term 1 election: Requested vote from peers fa544452588348449bc45d91f6b9cb73 (127.8.151.195:43259), ddfc05d6f35143cab6a47186b9967b77 (127.8.151.193:38439)
I20260928 06:44:27.562871 27902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e92af4cc82f44c589dc14bf20106b0a" candidate_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa544452588348449bc45d91f6b9cb73"
I20260928 06:44:27.562901 27640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e92af4cc82f44c589dc14bf20106b0a" candidate_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddfc05d6f35143cab6a47186b9967b77"
I20260928 06:44:27.562994 27902 raft_consensus.cc:3037] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:27.562994 27640 raft_consensus.cc:3037] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77 [term 0 FOLLOWER]: Advancing to term 1
W20260928 06:44:27.563390 27950 tablet.cc:2369] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:44:27.564213 27902 raft_consensus.cc:2454] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b2b7447d2f244c592e37ed1b3de6bc6 in term 1.
I20260928 06:44:27.564213 27640 raft_consensus.cc:2454] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b2b7447d2f244c592e37ed1b3de6bc6 in term 1.
I20260928 06:44:27.564519 27706 leader_election.cc:302] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b2b7447d2f244c592e37ed1b3de6bc6, ddfc05d6f35143cab6a47186b9967b77; no voters:
I20260928 06:44:27.564682 27973 raft_consensus.cc:2781] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:27.564895 27973 raft_consensus.cc:686] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 1 LEADER]: Becoming Leader. State: Replica: 9b2b7447d2f244c592e37ed1b3de6bc6, State: Running, Role: LEADER
I20260928 06:44:27.565210 27973 consensus_queue.cc:237] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 } } peers { permanent_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36259 } } peers { permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 } }
I20260928 06:44:27.568485 27497 catalog_manager.cc:5495] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b2b7447d2f244c592e37ed1b3de6bc6 (127.8.151.194). New cstate: current_term: 1 leader_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b2b7447d2f244c592e37ed1b3de6bc6" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 36259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 } health_report { overall_health: UNKNOWN } } }
W20260928 06:44:27.573976 27818 tablet.cc:2369] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:44:27.599074 27963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.595316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.8.151.194:36259)
0928 06:44:27.595456 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:27.595461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:27.595479 (+ 18us) server_negotiation.cc:407] Connection header received
0928 06:44:27.595586 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:27.595589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:27.595652 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:27.595758 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:27.596075 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.596629 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.597489 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:27.597764 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:27.597852 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:27.598205 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:27.598615 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:27.598749 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:27.598879 (+ 130us) server_negotiation.cc:299] Negotiation successful
0928 06:44:27.598916 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 06:44:27.606179 27902 raft_consensus.cc:1261] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73 [term 1 FOLLOWER]: Refusing update from remote peer 9b2b7447d2f244c592e37ed1b3de6bc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:27.606612 27640 raft_consensus.cc:1261] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77 [term 1 FOLLOWER]: Refusing update from remote peer 9b2b7447d2f244c592e37ed1b3de6bc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:27.607461 27976 consensus_queue.cc:1035] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfc05d6f35143cab6a47186b9967b77" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:27.607898 27976 consensus_queue.cc:1035] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa544452588348449bc45d91f6b9cb73" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 43259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:27.626418 27497 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:27.626576 27497 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:27.628923 27497 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84c19f71eb25404794bac200257c2eb6: Sending DeleteTablet for 3 replicas of tablet 7e92af4cc82f44c589dc14bf20106b0a
I20260928 06:44:27.629571 27882 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e92af4cc82f44c589dc14bf20106b0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:27 UTC) from {username='slave'} at 127.0.0.1:39838
I20260928 06:44:27.629624 27619 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e92af4cc82f44c589dc14bf20106b0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:27 UTC) from {username='slave'} at 127.0.0.1:42568
I20260928 06:44:27.629627 27751 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e92af4cc82f44c589dc14bf20106b0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:27 UTC) from {username='slave'} at 127.0.0.1:35552
I20260928 06:44:27.630005 27989 tablet_replica.cc:330] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: stopping tablet replica
I20260928 06:44:27.630049 27990 tablet_replica.cc:330] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77: stopping tablet replica
I20260928 06:44:27.630306 27990 raft_consensus.cc:2229] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:44:27.630442 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 27557
I20260928 06:44:27.630306 27989 raft_consensus.cc:2229] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:44:27.630620 27990 raft_consensus.cc:2258] T 7e92af4cc82f44c589dc14bf20106b0a P ddfc05d6f35143cab6a47186b9967b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:27.630666 27989 raft_consensus.cc:2258] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:27.631065 27991 tablet_replica.cc:330] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: stopping tablet replica
I20260928 06:44:27.631392 27991 raft_consensus.cc:2229] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:44:27.631744 27991 raft_consensus.cc:2258] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:27.632916 27991 ts_tablet_manager.cc:1905] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:44:27.635301 27991 ts_tablet_manager.cc:1918] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:44:27.635390 27991 log.cc:1196] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/7e92af4cc82f44c589dc14bf20106b0a
I20260928 06:44:27.635677 27991 ts_tablet_manager.cc:1939] T 7e92af4cc82f44c589dc14bf20106b0a P 9b2b7447d2f244c592e37ed1b3de6bc6: Deleting consensus metadata
I20260928 06:44:27.636286 27989 ts_tablet_manager.cc:1905] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:44:27.638727 27989 ts_tablet_manager.cc:1918] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:44:27.638818 27989 log.cc:1196] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/7e92af4cc82f44c589dc14bf20106b0a
W20260928 06:44:27.639009 27482 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:38439 (error 108)
I20260928 06:44:27.639111 27989 ts_tablet_manager.cc:1939] T 7e92af4cc82f44c589dc14bf20106b0a P fa544452588348449bc45d91f6b9cb73: Deleting consensus metadata
I20260928 06:44:27.639633 27483 catalog_manager.cc:4841] TS 9b2b7447d2f244c592e37ed1b3de6bc6 (127.8.151.194:36259): tablet 7e92af4cc82f44c589dc14bf20106b0a (table test-table [id=eed651f191d242068e126125c6eaae1b]) successfully deleted
I20260928 06:44:27.640100 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 27689
I20260928 06:44:27.640256 27480 catalog_manager.cc:4841] TS fa544452588348449bc45d91f6b9cb73 (127.8.151.195:43259): tablet 7e92af4cc82f44c589dc14bf20106b0a (table test-table [id=eed651f191d242068e126125c6eaae1b]) successfully deleted
I20260928 06:44:27.640794 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:27.640193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:38439 (local address 127.0.0.1:42574)
0928 06:44:27.640659 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:27.640726 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:38439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374}
W20260928 06:44:27.641073 27482 catalog_manager.cc:4584] TS ddfc05d6f35143cab6a47186b9967b77 (127.8.151.193:38439): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e92af4cc82f44c589dc14bf20106b0a: Network error: Client connection negotiation failed: client connection to 127.8.151.193:38439: connect: Connection refused (error 111)
I20260928 06:44:27.645354 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 27820
I20260928 06:44:27.650463 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 27465
2026-09-28T06:44:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1278 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:27 dist-test-slave-gtr8 krb5kdc[27995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:27 dist-test-slave-gtr8 krb5kdc[27995](info): set up 2 sockets
Sep 28 06:44:27 dist-test-slave-gtr8 krb5kdc[27995](info): commencing operation
krb5kdc: starting...
W20260928 06:44:28.679844 8799 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:28 dist-test-slave-gtr8 krb5kdc[27995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:28.802408 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39729
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:39729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:28.910239 28009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:28.910537 28009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:28.910595 28009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:28.914642 28009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:28.914706 28009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:28.914733 28009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:28.914753 28009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:28.914774 28009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:28.920035 28009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:39729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:39729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:28.921077 28009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:28.922112 28009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:28.930266 28014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:28.930287 28015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:28.930622 28017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:28.931105 28009 server_base.cc:1034] running on GCE node
I20260928 06:44:28.931677 28009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:28.932763 28009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:28.933930 28009 hybrid_clock.cc:648] HybridClock initialized: now 1790577868933898 us; error 54 us; skew 500 ppm
Sep 28 06:44:28 dist-test-slave-gtr8 krb5kdc[27995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577868, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:28.936770 28009 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:28.937678 28009 webserver.cc:460] Webserver started at http://127.8.151.254:42625/ using document root <none> and password file <none>
I20260928 06:44:28.938206 28009 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:28.938271 28009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:28.938493 28009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:28.940703 28009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "da7dea7232904c91b7ac85c88e6dfdf3"
format_stamp: "Formatted at 2026-09-28 06:44:28 on dist-test-slave-gtr8"
server_key: "081d42337968fbbd6b48ef66b28aec1b"
server_key_iv: "707251f3dabbbf232af4ffceb28ecc2e"
server_key_version: "encryptionkey@0"
I20260928 06:44:28.941231 28009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "da7dea7232904c91b7ac85c88e6dfdf3"
format_stamp: "Formatted at 2026-09-28 06:44:28 on dist-test-slave-gtr8"
server_key: "081d42337968fbbd6b48ef66b28aec1b"
server_key_iv: "707251f3dabbbf232af4ffceb28ecc2e"
server_key_version: "encryptionkey@0"
I20260928 06:44:28.944877 28009 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260928 06:44:28.947235 28024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:28.948577 28009 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260928 06:44:28.948689 28009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "da7dea7232904c91b7ac85c88e6dfdf3"
format_stamp: "Formatted at 2026-09-28 06:44:28 on dist-test-slave-gtr8"
server_key: "081d42337968fbbd6b48ef66b28aec1b"
server_key_iv: "707251f3dabbbf232af4ffceb28ecc2e"
server_key_version: "encryptionkey@0"
I20260928 06:44:28.948783 28009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:28.970619 28009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:28.973598 28009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:28.973768 28009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:28.980492 28009 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:39729
I20260928 06:44:28.980531 28076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:39729 every 8 connection(s)
I20260928 06:44:28.981376 28009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:28.983964 28077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:28.989028 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 28009
I20260928 06:44:28.989068 28077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3: Bootstrap starting.
I20260928 06:44:28.989166 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:28.989444 8799 external_mini_cluster.cc:1428] Setting key 223768195342d1974162c54c98a0c631
I20260928 06:44:28.991590 28077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:28.992424 28077 log.cc:824] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:28.994594 28077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3: No bootstrap required, opened a new log
Sep 28 06:44:28 dist-test-slave-gtr8 krb5kdc[27995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:28.998898 28077 raft_consensus.cc:348] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7dea7232904c91b7ac85c88e6dfdf3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39729 } }
I20260928 06:44:28.999121 28077 raft_consensus.cc:374] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:28.999200 28077 raft_consensus.cc:729] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7dea7232904c91b7ac85c88e6dfdf3, State: Initialized, Role: FOLLOWER
I20260928 06:44:28.999608 28077 consensus_queue.cc:260] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7dea7232904c91b7ac85c88e6dfdf3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39729 } }
I20260928 06:44:28.999739 28077 raft_consensus.cc:388] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:28.999801 28077 raft_consensus.cc:482] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:28.999898 28077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:29.000968 28077 raft_consensus.cc:504] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7dea7232904c91b7ac85c88e6dfdf3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39729 } }
I20260928 06:44:29.001307 28077 leader_election.cc:302] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da7dea7232904c91b7ac85c88e6dfdf3; no voters:
I20260928 06:44:29.001652 28077 leader_election.cc:288] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:29.001974 28082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:29.002168 28082 raft_consensus.cc:686] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [term 1 LEADER]: Becoming Leader. State: Replica: da7dea7232904c91b7ac85c88e6dfdf3, State: Running, Role: LEADER
I20260928 06:44:29.002462 28082 consensus_queue.cc:237] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7dea7232904c91b7ac85c88e6dfdf3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39729 } }
I20260928 06:44:29.002959 28077 sys_catalog.cc:564] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:29.003638 28082 sys_catalog.cc:455] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da7dea7232904c91b7ac85c88e6dfdf3. Latest consensus state: current_term: 1 leader_uuid: "da7dea7232904c91b7ac85c88e6dfdf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7dea7232904c91b7ac85c88e6dfdf3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39729 } } }
I20260928 06:44:29.003751 28082 sys_catalog.cc:458] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:29.004019 28084 sys_catalog.cc:455] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da7dea7232904c91b7ac85c88e6dfdf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da7dea7232904c91b7ac85c88e6dfdf3" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 39729 } } }
I20260928 06:44:29.004105 28084 sys_catalog.cc:458] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:29.004143 28090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:29.005089 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:28.990754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.8.151.254:39729)
0928 06:44:28.991213 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:28.991233 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:28.991281 (+ 48us) server_negotiation.cc:407] Connection header received
0928 06:44:28.992133 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:28.992155 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:28.992572 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:28.992918 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:28.993688 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:28.994648 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:28.995575 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:28.996023 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:28.996164 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:28.998940 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:28.998946 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:28.998963 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:28.998994 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:29.000934 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:29.001402 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:29.001408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:29.001414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:29.001547 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:29.001869 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:29.001873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:29.001875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:29.002463 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:29.003366 (+ 903us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.003522 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.003969 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":137,"threads_started":1}
I20260928 06:44:29.007346 28090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:29.013162 28090 catalog_manager.cc:1347] Generated new cluster ID: 8ef420c009904d449f29c54619a8b3bd
I20260928 06:44:29.013227 28090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:29.018815 28090 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:29.019752 28090 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:29.026471 28090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3: Generated new TSK 0
I20260928 06:44:29.027108 28090 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.151.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
Entry for principal kudu/127.8.151.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab.
I20260928 06:44:29.056423 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39729
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:29.198585 28103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:29.198889 28103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:29.198951 28103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:29.203197 28103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:29.203284 28103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:29.203370 28103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:44:29.208988 28103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.193.keytab
--principal=kudu/127.8.151.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:29.210088 28103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:29.211117 28103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:44:29.221714 28103 server_base.cc:1034] running on GCE node
W20260928 06:44:29.221664 28108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:29.221657 28109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:29.221920 28111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:29.222433 28103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:29.223070 28103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:29.224236 28103 hybrid_clock.cc:648] HybridClock initialized: now 1790577869224209 us; error 45 us; skew 500 ppm
Sep 28 06:44:29 dist-test-slave-gtr8 krb5kdc[27995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577869, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:29.226972 28103 init.cc:377] Logged in from keytab as kudu/127.8.151.193@KRBTEST.COM (short username kudu)
I20260928 06:44:29.227883 28103 webserver.cc:460] Webserver started at http://127.8.151.193:41009/ using document root <none> and password file <none>
I20260928 06:44:29.228421 28103 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:29.228488 28103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:29.228710 28103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:29.230756 28103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "ceb31109de314be2989d99f48528115a"
format_stamp: "Formatted at 2026-09-28 06:44:29 on dist-test-slave-gtr8"
server_key: "0b6159339f749487a96504f0533aad9a"
server_key_iv: "def8787e3004a234bcd695f90a097b17"
server_key_version: "encryptionkey@0"
I20260928 06:44:29.231360 28103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "ceb31109de314be2989d99f48528115a"
format_stamp: "Formatted at 2026-09-28 06:44:29 on dist-test-slave-gtr8"
server_key: "0b6159339f749487a96504f0533aad9a"
server_key_iv: "def8787e3004a234bcd695f90a097b17"
server_key_version: "encryptionkey@0"
I20260928 06:44:29.235186 28103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260928 06:44:29.237654 28118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:29.238948 28103 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:44:29.239066 28103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "ceb31109de314be2989d99f48528115a"
format_stamp: "Formatted at 2026-09-28 06:44:29 on dist-test-slave-gtr8"
server_key: "0b6159339f749487a96504f0533aad9a"
server_key_iv: "def8787e3004a234bcd695f90a097b17"
server_key_version: "encryptionkey@0"
I20260928 06:44:29.239223 28103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:29.276131 28103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:29.279589 28103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:29.279790 28103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:29.280431 28103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:29.281330 28103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:29.281385 28103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:29.281445 28103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:29.281474 28103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:29.291626 28103 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:39371
I20260928 06:44:29.291661 28232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:39371 every 8 connection(s)
I20260928 06:44:29.292570 28103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:44:29.293756 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 28103
I20260928 06:44:29.293882 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:44:29.294184 8799 external_mini_cluster.cc:1428] Setting key 214b7319b55ebead834f2eda791087b0
Sep 28 06:44:29 dist-test-slave-gtr8 krb5kdc[27995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577869, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.193@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.151.194@KRBTEST.COM; defaulting to no policy
I20260928 06:44:29.307770 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.294626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:37791 (local address 127.8.151.254:39729)
0928 06:44:29.294760 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:29.294765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:29.295610 (+ 845us) server_negotiation.cc:407] Connection header received
0928 06:44:29.296727 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:29.296730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:29.296785 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:29.296867 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:29.298225 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.298985 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.300720 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.300986 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.301075 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.303759 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:29.303764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:29.303768 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:29.303796 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:29.305410 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:29.306007 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:29.306012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:29.306013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:29.306106 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:29.306497 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:29.306500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:29.306502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:29.306726 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:29.306818 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.307450 (+ 632us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.307592 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.194@KRBTEST.COM" created.
I20260928 06:44:29.308707 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.294845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39729 (local address 127.8.151.193:37791)
0928 06:44:29.295423 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:29.295486 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:29.296504 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:29.297041 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:29.297050 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:29.297408 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:29.298047 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.298074 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.299218 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.299222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.300554 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.300567 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.301261 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.301264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.301414 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.301992 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:29.302010 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:29.303575 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:29.305522 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:29.305529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:29.305553 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:29.305881 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:29.306222 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:29.306225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:29.306227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:29.306388 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:29.306819 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:29.306824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:29.307193 (+ 369us) client_negotiation.cc:769] Sending connection context
0928 06:44:29.307368 (+ 175us) client_negotiation.cc:240] Negotiation successful
0928 06:44:29.307696 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":135,"threads_started":1}
I20260928 06:44:29.310066 28233 heartbeater.cc:347] Connected to a master server at 127.8.151.254:39729
I20260928 06:44:29.310351 28233 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:29.310942 28233 heartbeater.cc:511] Master 127.8.151.254:39729 requested a full tablet report, sending...
I20260928 06:44:29.312465 28041 ts_manager.cc:194] Registered new tserver with Master: ceb31109de314be2989d99f48528115a (127.8.151.193:39371)
I20260928 06:44:29.313602 28041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.193@KRBTEST.COM'} at 127.8.151.193:37791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
Entry for principal kudu/127.8.151.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab.
I20260928 06:44:29.326489 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39729
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:29.438858 28238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:29.439101 28238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:29.439172 28238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:29.443266 28238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:29.443323 28238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:29.443387 28238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:44:29.448727 28238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.194.keytab
--principal=kudu/127.8.151.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:29.449759 28238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:29.450737 28238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:29.460142 28244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:29.460217 28243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:29.460153 28246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:29.460961 28238 server_base.cc:1034] running on GCE node
I20260928 06:44:29.461396 28238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:29.462009 28238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:29.463217 28238 hybrid_clock.cc:648] HybridClock initialized: now 1790577869463189 us; error 44 us; skew 500 ppm
Sep 28 06:44:29 dist-test-slave-gtr8 krb5kdc[27995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577869, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:29.466037 28238 init.cc:377] Logged in from keytab as kudu/127.8.151.194@KRBTEST.COM (short username kudu)
I20260928 06:44:29.466956 28238 webserver.cc:460] Webserver started at http://127.8.151.194:46091/ using document root <none> and password file <none>
I20260928 06:44:29.467545 28238 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:29.467620 28238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:29.467825 28238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:29.469856 28238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "9bc022a7554341a284012e32245cbf73"
format_stamp: "Formatted at 2026-09-28 06:44:29 on dist-test-slave-gtr8"
server_key: "589be7320c7dc63e06b1fd0ae90dc602"
server_key_iv: "6e9549737e596e85ce7cc0220d253ac0"
server_key_version: "encryptionkey@0"
I20260928 06:44:29.470410 28238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "9bc022a7554341a284012e32245cbf73"
format_stamp: "Formatted at 2026-09-28 06:44:29 on dist-test-slave-gtr8"
server_key: "589be7320c7dc63e06b1fd0ae90dc602"
server_key_iv: "6e9549737e596e85ce7cc0220d253ac0"
server_key_version: "encryptionkey@0"
I20260928 06:44:29.474274 28238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:44:29.476737 28253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:29.477977 28238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:44:29.478092 28238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "9bc022a7554341a284012e32245cbf73"
format_stamp: "Formatted at 2026-09-28 06:44:29 on dist-test-slave-gtr8"
server_key: "589be7320c7dc63e06b1fd0ae90dc602"
server_key_iv: "6e9549737e596e85ce7cc0220d253ac0"
server_key_version: "encryptionkey@0"
I20260928 06:44:29.478202 28238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:29.496456 28238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:29.500890 28238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:29.501083 28238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:29.501816 28238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:29.502842 28238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:29.502907 28238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:29.502962 28238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:29.502991 28238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:29.513273 28238 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:42411
I20260928 06:44:29.513304 28367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:42411 every 8 connection(s)
I20260928 06:44:29.514201 28238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:44:29.523268 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 28238
I20260928 06:44:29.523384 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:44:29.523696 8799 external_mini_cluster.cc:1428] Setting key 72b1cd182657ec142c9bd720c327ec28
Sep 28 06:44:29 dist-test-slave-gtr8 krb5kdc[27995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577869, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.194@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:29.528978 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.516197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:57285 (local address 127.8.151.254:39729)
0928 06:44:29.516352 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:29.516357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:29.517239 (+ 882us) server_negotiation.cc:407] Connection header received
0928 06:44:29.518344 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:29.518347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:29.518394 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:29.518466 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:29.519885 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.520424 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.521929 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.522142 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.522216 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.524988 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:29.524991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:29.524994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:29.525019 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:29.526591 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:29.527338 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:29.527342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:29.527343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:29.527424 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:29.527773 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:29.527776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:29.527777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:29.527976 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:29.528046 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.528681 (+ 635us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.528808 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 06:44:29.529809 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.516564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39729 (local address 127.8.151.194:57285)
0928 06:44:29.517063 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:29.517120 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:29.518118 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:29.518650 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:29.518659 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:29.519109 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:29.519750 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.519768 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.520539 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.520542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.521812 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.521823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.522283 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.522286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.522476 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.523083 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:29.523102 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:29.524808 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:29.526838 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:29.526844 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:29.526859 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:29.527196 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:29.527519 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:29.527522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:29.527524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:29.527676 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:29.528113 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:29.528119 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:29.528395 (+ 276us) client_negotiation.cc:769] Sending connection context
0928 06:44:29.528570 (+ 175us) client_negotiation.cc:240] Negotiation successful
0928 06:44:29.528870 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":115,"threads_started":1}
I20260928 06:44:29.530953 28368 heartbeater.cc:347] Connected to a master server at 127.8.151.254:39729
I20260928 06:44:29.531247 28368 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:29.531785 28368 heartbeater.cc:511] Master 127.8.151.254:39729 requested a full tablet report, sending...
I20260928 06:44:29.532904 28041 ts_manager.cc:194] Registered new tserver with Master: 9bc022a7554341a284012e32245cbf73 (127.8.151.194:42411)
WARNING: no policy specified for kudu/127.8.151.195@KRBTEST.COM; defaulting to no policy
I20260928 06:44:29.533633 28041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.194@KRBTEST.COM'} at 127.8.151.194:57285
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
Entry for principal kudu/127.8.151.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab.
I20260928 06:44:29.551723 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39729
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:38721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:29.671165 28373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:29.671475 28373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:29.671558 28373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:29.675679 28373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:29.675746 28373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:29.675838 28373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:44:29.681118 28373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:38721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.195.keytab
--principal=kudu/127.8.151.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:39729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:29.682178 28373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:29.683224 28373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:29.692535 28381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:29.692538 28378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:29.692587 28379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:29.692852 28373 server_base.cc:1034] running on GCE node
I20260928 06:44:29.693225 28373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:29.693851 28373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:29.695003 28373 hybrid_clock.cc:648] HybridClock initialized: now 1790577869694980 us; error 41 us; skew 500 ppm
Sep 28 06:44:29 dist-test-slave-gtr8 krb5kdc[27995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577869, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:29.697559 28373 init.cc:377] Logged in from keytab as kudu/127.8.151.195@KRBTEST.COM (short username kudu)
I20260928 06:44:29.698431 28373 webserver.cc:460] Webserver started at http://127.8.151.195:40705/ using document root <none> and password file <none>
I20260928 06:44:29.698937 28373 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:29.698992 28373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:29.699177 28373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:29.701148 28373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "3c7548ee3e8745eaa84b482d00fe94ce"
format_stamp: "Formatted at 2026-09-28 06:44:29 on dist-test-slave-gtr8"
server_key: "c98235c24042e17daa5c0c178d1f3b19"
server_key_iv: "72d6f7d10714d8044b464049c06b53f0"
server_key_version: "encryptionkey@0"
I20260928 06:44:29.701624 28373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "3c7548ee3e8745eaa84b482d00fe94ce"
format_stamp: "Formatted at 2026-09-28 06:44:29 on dist-test-slave-gtr8"
server_key: "c98235c24042e17daa5c0c178d1f3b19"
server_key_iv: "72d6f7d10714d8044b464049c06b53f0"
server_key_version: "encryptionkey@0"
I20260928 06:44:29.705245 28373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260928 06:44:29.707545 28388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:29.709066 28373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:29.709201 28373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "3c7548ee3e8745eaa84b482d00fe94ce"
format_stamp: "Formatted at 2026-09-28 06:44:29 on dist-test-slave-gtr8"
server_key: "c98235c24042e17daa5c0c178d1f3b19"
server_key_iv: "72d6f7d10714d8044b464049c06b53f0"
server_key_version: "encryptionkey@0"
I20260928 06:44:29.709323 28373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:29.732122 28373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:29.736191 28373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:29.736399 28373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:29.737104 28373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:29.738093 28373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:29.738154 28373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:29.738204 28373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:29.738232 28373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:29.749650 28373 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:40917
I20260928 06:44:29.749712 28502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:40917 every 8 connection(s)
I20260928 06:44:29.750782 28373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:44:29.758473 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 28373
I20260928 06:44:29.758652 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:44:29.758922 8799 external_mini_cluster.cc:1428] Setting key e3a81fe86a68cb578076263da7351133
Sep 28 06:44:29 dist-test-slave-gtr8 krb5kdc[27995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577869, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.195@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:29.765697 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.752881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:52363 (local address 127.8.151.254:39729)
0928 06:44:29.753034 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:29.753038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:29.753945 (+ 907us) server_negotiation.cc:407] Connection header received
0928 06:44:29.755620 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:29.755624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:29.755673 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:29.755749 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:29.757177 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.757730 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.759472 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.759758 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.759856 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.762503 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:29.762507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:29.762510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:29.762536 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:29.763811 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:29.764269 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:29.764272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:29.764273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:29.764346 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:29.764659 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:29.764662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:29.764664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:29.764830 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:29.764911 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.765429 (+ 518us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.765541 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 06:44:29.766463 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.753247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:39729 (local address 127.8.151.195:52363)
0928 06:44:29.753779 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:29.753835 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:29.755377 (+ 1542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:29.755927 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:29.755935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:29.756403 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:29.757015 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.757032 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.757859 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.757863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.759314 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.759325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.760095 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.760099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.760247 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.760808 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:29.760824 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:29.762351 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:29.763919 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:29.763925 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:29.763939 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:29.764176 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:29.764428 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:29.764430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:29.764432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:29.764564 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:29.764927 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:29.764953 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:29.765207 (+ 254us) client_negotiation.cc:769] Sending connection context
0928 06:44:29.765370 (+ 163us) client_negotiation.cc:240] Negotiation successful
0928 06:44:29.765682 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":148,"threads_started":1}
I20260928 06:44:29.767639 28503 heartbeater.cc:347] Connected to a master server at 127.8.151.254:39729
I20260928 06:44:29.767877 28503 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:29.768357 28503 heartbeater.cc:511] Master 127.8.151.254:39729 requested a full tablet report, sending...
I20260928 06:44:29.769362 28041 ts_manager.cc:194] Registered new tserver with Master: 3c7548ee3e8745eaa84b482d00fe94ce (127.8.151.195:40917)
I20260928 06:44:29.770084 28041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.151.195@KRBTEST.COM'} at 127.8.151.195:52363
I20260928 06:44:29.772693 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:44:29.782418 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.774856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57936 (local address 127.8.151.254:39729)
0928 06:44:29.774983 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:29.774987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:29.775121 (+ 134us) server_negotiation.cc:407] Connection header received
0928 06:44:29.775265 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:29.775267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:29.775303 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:29.775359 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:29.776205 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.776720 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.777911 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.778124 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.778200 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.779595 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:29.779598 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:29.779600 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:29.779620 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:29.780910 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:29.781278 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:29.781282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:29.781283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:29.781357 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:29.781593 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:29.781597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:29.781598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:29.781807 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:29.781919 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.782169 (+ 250us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.782273 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:44:29.785144 28041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:44:29.788849 28041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:44:29.804569 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.797294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:39371 (local address 127.0.0.1:47336)
0928 06:44:29.798611 (+ 1317us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:29.798660 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:29.798793 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:29.799654 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:29.799660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:29.799680 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:29.799823 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.799831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.801304 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.801307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.802728 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.802737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.804039 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.804042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.804162 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.804204 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 06:44:29.804299 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 06:44:29.804417 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1139,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260928 06:44:29.805547 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.797991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45390 (local address 127.8.151.194:42411)
0928 06:44:29.798329 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:29.798339 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:29.798771 (+ 432us) server_negotiation.cc:407] Connection header received
0928 06:44:29.799610 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:29.799616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:29.799818 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:29.799936 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:29.800380 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.802304 (+ 1924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.803470 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.804386 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.804487 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.804618 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.805304 (+ 686us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.805405 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":107,"threads_started":1}
I20260928 06:44:29.805636 28519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.798928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60318 (local address 127.8.151.195:40917)
0928 06:44:29.799283 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:29.799296 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:29.799316 (+ 20us) server_negotiation.cc:407] Connection header received
0928 06:44:29.799486 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:29.799492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:29.799674 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:29.799827 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:29.800208 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.801319 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.803901 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.804664 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.804757 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.804890 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.805425 (+ 535us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.805509 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":144,"threads_started":1}
I20260928 06:44:29.805653 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.797428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.8.151.193:39371)
0928 06:44:29.797673 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:29.797689 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:29.798937 (+ 1248us) server_negotiation.cc:407] Connection header received
0928 06:44:29.799027 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:29.799032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:29.799309 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:29.799457 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:29.799951 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.801147 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.802854 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.803724 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.803835 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.805376 (+ 1541us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.805427 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.805519 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260928 06:44:29.805333 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.798406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:40917 (local address 127.0.0.1:60318)
0928 06:44:29.799253 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:29.799266 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:29.799337 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:29.799981 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:29.799983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:29.800001 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:29.800086 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.800091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.801451 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.801454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.803761 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.803781 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.804947 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.804950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.805044 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.805067 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:44:29.805135 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 06:44:29.805202 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":1122,"thread_start_us":96,"threads_started":1}
I20260928 06:44:29.806141 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.797352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42411 (local address 127.0.0.1:45390)
0928 06:44:29.798637 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:29.798679 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:29.799455 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:29.800143 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:29.800146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:29.800163 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:29.800245 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.800250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.802440 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.802443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.803361 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.803371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.805006 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.805009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.805143 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.805161 (+ 18us) client_negotiation.cc:769] Sending connection context
0928 06:44:29.805954 (+ 793us) client_negotiation.cc:240] Negotiation successful
0928 06:44:29.806028 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1222,"thread_start_us":81,"threads_started":1}
I20260928 06:44:29.808687 28302 tablet_service.cc:1467] Processing CreateTablet for tablet 9e12aab648d54668b1f7e18355cf6f92 (DEFAULT_TABLE table=test-table [id=ddc217917b4e4754b1b94484eaedc3e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:29.808854 28167 tablet_service.cc:1467] Processing CreateTablet for tablet 9e12aab648d54668b1f7e18355cf6f92 (DEFAULT_TABLE table=test-table [id=ddc217917b4e4754b1b94484eaedc3e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:29.808826 28436 tablet_service.cc:1467] Processing CreateTablet for tablet 9e12aab648d54668b1f7e18355cf6f92 (DEFAULT_TABLE table=test-table [id=ddc217917b4e4754b1b94484eaedc3e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:29.809794 28302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e12aab648d54668b1f7e18355cf6f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:29.809825 28167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e12aab648d54668b1f7e18355cf6f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:29.809832 28436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e12aab648d54668b1f7e18355cf6f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:29.817936 28520 tablet_bootstrap.cc:492] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a: Bootstrap starting.
I20260928 06:44:29.818130 28521 tablet_bootstrap.cc:492] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce: Bootstrap starting.
I20260928 06:44:29.819891 28520 tablet_bootstrap.cc:654] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:29.820539 28521 tablet_bootstrap.cc:654] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:29.820592 28520 log.cc:824] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:29.821332 28521 log.cc:824] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:29.821981 28522 tablet_bootstrap.cc:492] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73: Bootstrap starting.
I20260928 06:44:29.822431 28520 tablet_bootstrap.cc:492] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a: No bootstrap required, opened a new log
I20260928 06:44:29.822707 28520 ts_tablet_manager.cc:1397] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260928 06:44:29.824065 28521 tablet_bootstrap.cc:492] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce: No bootstrap required, opened a new log
I20260928 06:44:29.824302 28521 ts_tablet_manager.cc:1397] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:44:29.824348 28522 tablet_bootstrap.cc:654] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:29.825181 28522 log.cc:824] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:29.826967 28522 tablet_bootstrap.cc:492] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73: No bootstrap required, opened a new log
I20260928 06:44:29.827193 28522 ts_tablet_manager.cc:1397] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260928 06:44:29.827093 28520 raft_consensus.cc:348] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 } } peers { permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 } } peers { permanent_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 40917 } }
I20260928 06:44:29.827430 28520 raft_consensus.cc:374] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:29.827512 28520 raft_consensus.cc:729] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb31109de314be2989d99f48528115a, State: Initialized, Role: FOLLOWER
I20260928 06:44:29.828008 28520 consensus_queue.cc:260] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 } } peers { permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 } } peers { permanent_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 40917 } }
I20260928 06:44:29.827994 28521 raft_consensus.cc:348] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 } } peers { permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 } } peers { permanent_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 40917 } }
I20260928 06:44:29.828274 28521 raft_consensus.cc:374] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:29.828367 28521 raft_consensus.cc:729] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7548ee3e8745eaa84b482d00fe94ce, State: Initialized, Role: FOLLOWER
I20260928 06:44:29.828843 28521 consensus_queue.cc:260] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 } } peers { permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 } } peers { permanent_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 40917 } }
I20260928 06:44:29.829047 28520 ts_tablet_manager.cc:1428] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:44:29.829442 28233 heartbeater.cc:503] Master 127.8.151.254:39729 was elected leader, sending a full tablet report...
I20260928 06:44:29.829493 28503 heartbeater.cc:503] Master 127.8.151.254:39729 was elected leader, sending a full tablet report...
I20260928 06:44:29.830680 28521 ts_tablet_manager.cc:1428] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:44:29.833909 28522 raft_consensus.cc:348] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 } } peers { permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 } } peers { permanent_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 40917 } }
I20260928 06:44:29.834110 28522 raft_consensus.cc:374] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:29.834179 28522 raft_consensus.cc:729] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc022a7554341a284012e32245cbf73, State: Initialized, Role: FOLLOWER
I20260928 06:44:29.834575 28522 consensus_queue.cc:260] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 } } peers { permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 } } peers { permanent_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 40917 } }
I20260928 06:44:29.835240 28368 heartbeater.cc:503] Master 127.8.151.254:39729 was elected leader, sending a full tablet report...
I20260928 06:44:29.835366 28522 ts_tablet_manager.cc:1428] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260928 06:44:29.865880 28527 raft_consensus.cc:482] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:44:29.866074 28527 raft_consensus.cc:504] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 } } peers { permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 } } peers { permanent_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 40917 } }
I20260928 06:44:29.867308 28527 leader_election.cc:288] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb31109de314be2989d99f48528115a (127.8.151.193:39371), 9bc022a7554341a284012e32245cbf73 (127.8.151.194:42411)
I20260928 06:44:29.871547 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.867832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:43957 (local address 127.8.151.194:42411)
0928 06:44:29.867953 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:29.867958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:29.868022 (+ 64us) server_negotiation.cc:407] Connection header received
0928 06:44:29.868166 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:29.868168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:29.868212 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:29.868274 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:29.868637 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.869268 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.870614 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.871012 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.871125 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.871200 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.871356 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.871426 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:44:29.871568 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.867538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42411 (local address 127.8.151.195:43957)
0928 06:44:29.867941 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:29.867955 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:29.868057 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:29.868372 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:29.868375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:29.868392 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:29.868517 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.868523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.869377 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.869380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.870511 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.870519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.871123 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.871127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.871247 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.871270 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:44:29.871335 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 06:44:29.871409 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":3,"thread_start_us":120,"threads_started":1}
I20260928 06:44:29.871670 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.867467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:39371 (local address 127.8.151.195:58343)
0928 06:44:29.867639 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:29.867654 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:29.867742 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:29.868267 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:29.868270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:29.868288 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:29.868408 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.868414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.869163 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.869166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.870364 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:29.870374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.871284 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.871287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:29.871394 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.871414 (+ 20us) client_negotiation.cc:769] Sending connection context
0928 06:44:29.871478 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 06:44:29.871547 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260928 06:44:29.872177 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.867921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:58343 (local address 127.8.151.193:39371)
0928 06:44:29.868047 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:29.868051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:29.868064 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:44:29.868117 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:29.868120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:29.868170 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:29.868254 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:29.868526 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.869059 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.870522 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.871043 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.871177 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.871224 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.872002 (+ 778us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.872087 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 06:44:29.872326 28322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e12aab648d54668b1f7e18355cf6f92" candidate_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc022a7554341a284012e32245cbf73" is_pre_election: true
I20260928 06:44:29.872648 28322 raft_consensus.cc:2454] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c7548ee3e8745eaa84b482d00fe94ce in term 0.
I20260928 06:44:29.872750 28187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e12aab648d54668b1f7e18355cf6f92" candidate_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb31109de314be2989d99f48528115a" is_pre_election: true
I20260928 06:44:29.873030 28187 raft_consensus.cc:2454] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c7548ee3e8745eaa84b482d00fe94ce in term 0.
I20260928 06:44:29.873126 28392 leader_election.cc:302] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c7548ee3e8745eaa84b482d00fe94ce, 9bc022a7554341a284012e32245cbf73; no voters:
I20260928 06:44:29.873425 28527 raft_consensus.cc:2781] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:44:29.873512 28527 raft_consensus.cc:482] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:44:29.873556 28527 raft_consensus.cc:3037] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:29.874778 28527 raft_consensus.cc:504] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 } } peers { permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 } } peers { permanent_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 40917 } }
I20260928 06:44:29.875128 28527 leader_election.cc:288] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [CANDIDATE]: Term 1 election: Requested vote from peers ceb31109de314be2989d99f48528115a (127.8.151.193:39371), 9bc022a7554341a284012e32245cbf73 (127.8.151.194:42411)
I20260928 06:44:29.875494 28187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e12aab648d54668b1f7e18355cf6f92" candidate_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb31109de314be2989d99f48528115a"
I20260928 06:44:29.875501 28322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e12aab648d54668b1f7e18355cf6f92" candidate_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc022a7554341a284012e32245cbf73"
I20260928 06:44:29.875612 28187 raft_consensus.cc:3037] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:29.875627 28322 raft_consensus.cc:3037] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:29.876672 28322 raft_consensus.cc:2454] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c7548ee3e8745eaa84b482d00fe94ce in term 1.
I20260928 06:44:29.876763 28187 raft_consensus.cc:2454] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c7548ee3e8745eaa84b482d00fe94ce in term 1.
I20260928 06:44:29.876992 28392 leader_election.cc:302] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c7548ee3e8745eaa84b482d00fe94ce, 9bc022a7554341a284012e32245cbf73; no voters:
I20260928 06:44:29.877398 28527 raft_consensus.cc:2781] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:29.877637 28527 raft_consensus.cc:686] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 1 LEADER]: Becoming Leader. State: Replica: 3c7548ee3e8745eaa84b482d00fe94ce, State: Running, Role: LEADER
I20260928 06:44:29.877980 28527 consensus_queue.cc:237] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 } } peers { permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 } } peers { permanent_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 40917 } }
I20260928 06:44:29.881392 28041 catalog_manager.cc:5495] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c7548ee3e8745eaa84b482d00fe94ce (127.8.151.195). New cstate: current_term: 1 leader_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c7548ee3e8745eaa84b482d00fe94ce" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 40917 } health_report { overall_health: HEALTHY } } }
I20260928 06:44:29.904856 28519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.900894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.8.151.195:40917)
0928 06:44:29.901019 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:29.901023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:29.901038 (+ 15us) server_negotiation.cc:407] Connection header received
0928 06:44:29.901178 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:29.901181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:29.901242 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:29.901346 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:29.901530 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.902234 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.903073 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:29.903400 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:29.903497 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:29.903747 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:29.904256 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:29.904365 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:29.904614 (+ 249us) server_negotiation.cc:299] Negotiation successful
0928 06:44:29.904721 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 06:44:29.911031 28187 raft_consensus.cc:1261] T 9e12aab648d54668b1f7e18355cf6f92 P ceb31109de314be2989d99f48528115a [term 1 FOLLOWER]: Refusing update from remote peer 3c7548ee3e8745eaa84b482d00fe94ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:29.911042 28322 raft_consensus.cc:1261] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73 [term 1 FOLLOWER]: Refusing update from remote peer 3c7548ee3e8745eaa84b482d00fe94ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:29.911744 28527 consensus_queue.cc:1035] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb31109de314be2989d99f48528115a" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 39371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:29.911952 28530 consensus_queue.cc:1035] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc022a7554341a284012e32245cbf73" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 42411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:29.918596 28533 mvcc.cc:205] Tried to move back new op lower bound from 7334206955147862016 to 7334206955023081472. Current Snapshot: MvccSnapshot[applied={T|T < 7334206955147862016}]
I20260928 06:44:29.921977 28536 mvcc.cc:205] Tried to move back new op lower bound from 7334206955147862016 to 7334206955023081472. Current Snapshot: MvccSnapshot[applied={T|T < 7334206955147862016}]
I20260928 06:44:29.925292 28534 mvcc.cc:205] Tried to move back new op lower bound from 7334206955147862016 to 7334206955023081472. Current Snapshot: MvccSnapshot[applied={T|T < 7334206955147862016}]
I20260928 06:44:29.927836 28041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:29.928001 28041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:29.930445 28041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da7dea7232904c91b7ac85c88e6dfdf3: Sending DeleteTablet for 3 replicas of tablet 9e12aab648d54668b1f7e18355cf6f92
I20260928 06:44:29.931280 28436 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e12aab648d54668b1f7e18355cf6f92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:29 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:60318
I20260928 06:44:29.931440 28167 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e12aab648d54668b1f7e18355cf6f92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:29 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:47336
I20260928 06:44:29.931439 28302 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e12aab648d54668b1f7e18355cf6f92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:29 UTC) from {username='kudu', principal='kudu/127.8.151.254@KRBTEST.COM'} at 127.0.0.1:45390
I20260928 06:44:29.932366 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 28103
I20260928 06:44:29.939307 28545 tablet_replica.cc:330] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73: stopping tablet replica
I20260928 06:44:29.939653 28545 raft_consensus.cc:2229] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:44:29.939981 28545 raft_consensus.cc:2258] T 9e12aab648d54668b1f7e18355cf6f92 P 9bc022a7554341a284012e32245cbf73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:29.939123 28543 tablet_replica.cc:330] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce: stopping tablet replica
I20260928 06:44:29.940481 28543 raft_consensus.cc:2229] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:44:29.940816 28543 raft_consensus.cc:2258] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 06:44:29.940936 28026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:39371 (error 108)
I20260928 06:44:29.941735 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 28238
I20260928 06:44:29.948187 28543 ts_tablet_manager.cc:1905] T 9e12aab648d54668b1f7e18355cf6f92 P 3c7548ee3e8745eaa84b482d00fe94ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:44:29.949402 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 28373
I20260928 06:44:29.951447 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.950926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:39371 (local address 127.0.0.1:47346)
0928 06:44:29.951290 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:29.951358 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285}
I20260928 06:44:29.951678 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:29.951128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:42411 (local address 127.0.0.1:45404)
0928 06:44:29.951574 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:29.951615 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.194:42411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401}
W20260928 06:44:29.951709 28026 catalog_manager.cc:4584] TS ceb31109de314be2989d99f48528115a (127.8.151.193:39371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9e12aab648d54668b1f7e18355cf6f92: Network error: Client connection negotiation failed: client connection to 127.8.151.193:39371: connect: Connection refused (error 111)
I20260928 06:44:29.956259 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 28009
2026-09-28T06:44:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2306 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-28T06:44:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:29Z Disabled control of system clock
I20260928 06:44:29.986414 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:35027
--webserver_interface=127.8.151.254
--webserver_port=0
--builtin_ntp_servers=127.8.151.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:35027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 06:44:30.093151 28550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:30.093400 28550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:30.093451 28550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:30.097355 28550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:30.097410 28550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:30.097431 28550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:30.097452 28550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:30.097477 28550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:30.102213 28550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:35027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:35027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:30.103246 28550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:30.104547 28550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:30.112564 28555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:30.112668 28558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:30.112591 28556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:30.113178 28550 server_base.cc:1034] running on GCE node
I20260928 06:44:30.113734 28550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:30.114811 28550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:30.116003 28550 hybrid_clock.cc:648] HybridClock initialized: now 1790577870115978 us; error 35 us; skew 500 ppm
I20260928 06:44:30.117837 28550 webserver.cc:460] Webserver started at http://127.8.151.254:43003/ using document root <none> and password file <none>
I20260928 06:44:30.118376 28550 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:30.118435 28550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:30.118615 28550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:30.120620 28550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "77c3435c9684439da66603c36d15c6ab"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "3fdaa2c0475323e6254fcf5fe312bb2e"
server_key_iv: "6a1652685af8a2f1a0fb549a37e88f80"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.121156 28550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "77c3435c9684439da66603c36d15c6ab"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "3fdaa2c0475323e6254fcf5fe312bb2e"
server_key_iv: "6a1652685af8a2f1a0fb549a37e88f80"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.124686 28550 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260928 06:44:30.126901 28564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:30.128269 28550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:44:30.128369 28550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "77c3435c9684439da66603c36d15c6ab"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "3fdaa2c0475323e6254fcf5fe312bb2e"
server_key_iv: "6a1652685af8a2f1a0fb549a37e88f80"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.128472 28550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:30.148588 28550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:30.149304 28550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:30.149475 28550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:30.156633 28550 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:35027
I20260928 06:44:30.156785 28617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:35027 every 8 connection(s)
I20260928 06:44:30.157608 28550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:30.160703 28618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:30.161883 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 28550
I20260928 06:44:30.162017 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:30.162321 8799 external_mini_cluster.cc:1428] Setting key 15f088ea6d7909cc0f65e575c9389104
I20260928 06:44:30.167272 28618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab: Bootstrap starting.
I20260928 06:44:30.169780 28618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:30.170490 28618 log.cc:824] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:30.172514 28618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab: No bootstrap required, opened a new log
I20260928 06:44:30.174196 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.163866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39766 (local address 127.8.151.254:35027)
0928 06:44:30.164634 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:30.164658 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:30.164705 (+ 47us) server_negotiation.cc:407] Connection header received
0928 06:44:30.165603 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:30.165629 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:30.166201 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:30.166601 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:30.167096 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.168636 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.169838 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.170543 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.170744 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.171706 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:30.171713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:30.171730 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:30.172250 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:30.172321 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:30.172347 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:30.172441 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:30.172618 (+ 177us) server_negotiation.cc:299] Negotiation successful
0928 06:44:30.173111 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":139,"threads_started":1}
I20260928 06:44:30.176196 28618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c3435c9684439da66603c36d15c6ab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } }
I20260928 06:44:30.176409 28618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:30.176465 28618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77c3435c9684439da66603c36d15c6ab, State: Initialized, Role: FOLLOWER
I20260928 06:44:30.176898 28618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c3435c9684439da66603c36d15c6ab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } }
I20260928 06:44:30.177047 28618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:30.177109 28618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:30.177199 28618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:30.178025 28618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c3435c9684439da66603c36d15c6ab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } }
I20260928 06:44:30.178364 28618 leader_election.cc:302] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77c3435c9684439da66603c36d15c6ab; no voters:
I20260928 06:44:30.178699 28618 leader_election.cc:288] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:30.178757 28623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:30.178942 28623 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [term 1 LEADER]: Becoming Leader. State: Replica: 77c3435c9684439da66603c36d15c6ab, State: Running, Role: LEADER
I20260928 06:44:30.179262 28623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c3435c9684439da66603c36d15c6ab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } }
I20260928 06:44:30.179948 28618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:30.180778 28624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77c3435c9684439da66603c36d15c6ab. Latest consensus state: current_term: 1 leader_uuid: "77c3435c9684439da66603c36d15c6ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c3435c9684439da66603c36d15c6ab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } } }
I20260928 06:44:30.180908 28624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:30.181206 28625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77c3435c9684439da66603c36d15c6ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c3435c9684439da66603c36d15c6ab" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 35027 } } }
I20260928 06:44:30.181293 28625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:30.182132 28632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:30.185060 28632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:30.190987 28632 catalog_manager.cc:1347] Generated new cluster ID: c527a889d03d4d9799e0cfea8acfc28f
I20260928 06:44:30.191049 28632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:30.209846 28632 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:30.211089 28632 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:30.225557 28632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab: Generated new TSK 0
I20260928 06:44:30.226465 28632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:30.239861 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.193:0
--local_ip_for_outbound_sockets=127.8.151.193
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--builtin_ntp_servers=127.8.151.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 06:44:30.381623 28642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:30.381878 28642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:30.381915 28642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:30.385838 28642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:30.385890 28642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:30.385941 28642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.193
I20260928 06:44:30.390544 28642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.151.193
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.193
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:30.391569 28642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:30.393085 28642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:30.402458 28650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:30.402450 28647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:30.402470 28648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:30.403016 28642 server_base.cc:1034] running on GCE node
I20260928 06:44:30.403411 28642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:30.404017 28642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:30.405189 28642 hybrid_clock.cc:648] HybridClock initialized: now 1790577870405162 us; error 37 us; skew 500 ppm
I20260928 06:44:30.406953 28642 webserver.cc:460] Webserver started at http://127.8.151.193:46307/ using document root <none> and password file <none>
I20260928 06:44:30.407519 28642 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:30.407578 28642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:30.407755 28642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:30.409734 28642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/data/instance:
uuid: "6859b87ea6054d6fab554cc3a73f9cf2"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "d9d6136dbf338cd59070f5404d4e253e"
server_key_iv: "206a4f5d298bceae2bfd445c23465d80"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.410250 28642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance:
uuid: "6859b87ea6054d6fab554cc3a73f9cf2"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "d9d6136dbf338cd59070f5404d4e253e"
server_key_iv: "206a4f5d298bceae2bfd445c23465d80"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.414027 28642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260928 06:44:30.416474 28656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:30.417744 28642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:44:30.417867 28642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/wal
uuid: "6859b87ea6054d6fab554cc3a73f9cf2"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "d9d6136dbf338cd59070f5404d4e253e"
server_key_iv: "206a4f5d298bceae2bfd445c23465d80"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.417963 28642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:30.469947 28642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:30.470674 28642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:30.470866 28642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:30.471498 28642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:30.472453 28642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:30.472502 28642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:30.472553 28642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:30.472579 28642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:30.482510 28642 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.193:43235
I20260928 06:44:30.482551 28769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.193:43235 every 8 connection(s)
I20260928 06:44:30.483516 28642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/data/info.pb
I20260928 06:44:30.487875 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 28642
I20260928 06:44:30.487977 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-0/wal/instance
I20260928 06:44:30.488232 8799 external_mini_cluster.cc:1428] Setting key f3fc39479519a6ffba5adf6a67640f14
I20260928 06:44:30.490103 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.194:0
--local_ip_for_outbound_sockets=127.8.151.194
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--builtin_ntp_servers=127.8.151.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 06:44:30.494107 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.485441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.193:45823 (local address 127.8.151.254:35027)
0928 06:44:30.485588 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:30.485592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:30.486709 (+ 1117us) server_negotiation.cc:407] Connection header received
0928 06:44:30.488452 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:30.488455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:30.488516 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:30.488615 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:30.489663 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.490402 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.491948 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.492272 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.492378 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.493594 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:30.493598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:30.493601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:30.493617 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:30.493644 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:30.493654 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:30.493733 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:30.493877 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 06:44:30.493950 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 06:44:30.495296 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.485806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:35027 (local address 127.8.151.193:45823)
0928 06:44:30.486544 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:30.486598 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:30.488223 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:30.488786 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:30.488794 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:30.488897 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:30.489521 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.489538 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.490540 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.490544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.491823 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.491834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.492425 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.492428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.492610 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.493269 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:44:30.493271 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:30.493324 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:44:30.493399 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:30.493739 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:30.493747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:30.493752 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 06:44:30.493863 (+ 111us) client_negotiation.cc:240] Negotiation successful
0928 06:44:30.494134 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
I20260928 06:44:30.496786 28770 heartbeater.cc:347] Connected to a master server at 127.8.151.254:35027
I20260928 06:44:30.497012 28770 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:30.497565 28770 heartbeater.cc:511] Master 127.8.151.254:35027 requested a full tablet report, sending...
I20260928 06:44:30.499121 28581 ts_manager.cc:194] Registered new tserver with Master: 6859b87ea6054d6fab554cc3a73f9cf2 (127.8.151.193:43235)
I20260928 06:44:30.500551 28581 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.193:45823
W20260928 06:44:30.598852 28773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:30.599097 28773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:30.599176 28773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:30.603188 28773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:30.603241 28773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:30.603320 28773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.194
I20260928 06:44:30.608150 28773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.151.194
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.194
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:30.609200 28773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:30.610560 28773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:30.619939 28778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:30.619982 28781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:30.619935 28779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:30.620428 28773 server_base.cc:1034] running on GCE node
I20260928 06:44:30.621047 28773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:30.621874 28773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:30.623067 28773 hybrid_clock.cc:648] HybridClock initialized: now 1790577870623025 us; error 50 us; skew 500 ppm
I20260928 06:44:30.629721 28773 webserver.cc:460] Webserver started at http://127.8.151.194:45181/ using document root <none> and password file <none>
I20260928 06:44:30.630456 28773 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:30.630543 28773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:30.630761 28773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:30.633545 28773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/data/instance:
uuid: "704059d24f87441c9a6af0b950b3c2d3"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "ffcd47871f147bdd5f030f0bb547e7d3"
server_key_iv: "1f1d988d687a7c14b24aabc6bb648514"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.634220 28773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance:
uuid: "704059d24f87441c9a6af0b950b3c2d3"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "ffcd47871f147bdd5f030f0bb547e7d3"
server_key_iv: "1f1d988d687a7c14b24aabc6bb648514"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.639109 28773 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260928 06:44:30.642222 28787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:30.643774 28773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260928 06:44:30.643893 28773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/wal
uuid: "704059d24f87441c9a6af0b950b3c2d3"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "ffcd47871f147bdd5f030f0bb547e7d3"
server_key_iv: "1f1d988d687a7c14b24aabc6bb648514"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.644012 28773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:30.669966 28773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:30.670842 28773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:30.671041 28773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:30.671805 28773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:30.672986 28773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:30.673048 28773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:30.673101 28773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:30.673131 28773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:30.684006 28773 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.194:46505
I20260928 06:44:30.684038 28902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.194:46505 every 8 connection(s)
I20260928 06:44:30.685137 28773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/data/info.pb
I20260928 06:44:30.686415 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 28773
I20260928 06:44:30.686527 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/wal/instance
I20260928 06:44:30.686820 8799 external_mini_cluster.cc:1428] Setting key d5e76dad353e51f7752925219f6dcdf9
I20260928 06:44:30.688875 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.151.195:0
--local_ip_for_outbound_sockets=127.8.151.195
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--builtin_ntp_servers=127.8.151.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 06:44:30.696269 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.686959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:41637 (local address 127.8.151.254:35027)
0928 06:44:30.687107 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:30.687112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:30.688465 (+ 1353us) server_negotiation.cc:407] Connection header received
0928 06:44:30.690253 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:30.690256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:30.690311 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:30.690395 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:30.691572 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.692345 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.693982 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.694318 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.694430 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.695717 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:30.695722 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:30.695725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:30.695741 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:30.695766 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:30.695777 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:30.695859 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:30.696030 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 06:44:30.696100 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 06:44:30.697274 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.687506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:35027 (local address 127.8.151.194:41637)
0928 06:44:30.688301 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:30.688356 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:30.689974 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:30.690576 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:30.690585 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:30.690697 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:30.691399 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.691419 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.692462 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.692466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.693854 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.693865 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.694488 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.694492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.694690 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.695411 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:44:30.695413 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:30.695467 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:44:30.695548 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:30.695868 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:30.695877 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:30.695883 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 06:44:30.696001 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 06:44:30.696297 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":127,"threads_started":1}
I20260928 06:44:30.698570 28903 heartbeater.cc:347] Connected to a master server at 127.8.151.254:35027
I20260928 06:44:30.698824 28903 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:30.699450 28903 heartbeater.cc:511] Master 127.8.151.254:35027 requested a full tablet report, sending...
I20260928 06:44:30.700553 28581 ts_manager.cc:194] Registered new tserver with Master: 704059d24f87441c9a6af0b950b3c2d3 (127.8.151.194:46505)
I20260928 06:44:30.701280 28581 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.194:41637
W20260928 06:44:30.816036 28906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:30.816411 28906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:30.816540 28906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:30.822540 28906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:30.822619 28906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:30.822702 28906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.151.195
I20260928 06:44:30.829922 28906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.151.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.151.195
--webserver_port=0
--tserver_master_addrs=127.8.151.254:35027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.151.195
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:30.831389 28906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:30.833261 28906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:30.843945 28911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:30.843951 28914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:30.844072 28906 server_base.cc:1034] running on GCE node
W20260928 06:44:30.844185 28912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:30.844720 28906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:30.845350 28906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:30.846515 28906 hybrid_clock.cc:648] HybridClock initialized: now 1790577870846483 us; error 53 us; skew 500 ppm
I20260928 06:44:30.848294 28906 webserver.cc:460] Webserver started at http://127.8.151.195:44159/ using document root <none> and password file <none>
I20260928 06:44:30.848868 28906 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:30.848935 28906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:30.849134 28906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:30.851243 28906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/data/instance:
uuid: "f0ef234bfba9453b8c0d047720c6d16c"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "ad722cde1198c6052b88d29391c0ec4c"
server_key_iv: "a769af82f34ce2f919cf3bb6eec9d85c"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.851774 28906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance:
uuid: "f0ef234bfba9453b8c0d047720c6d16c"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "ad722cde1198c6052b88d29391c0ec4c"
server_key_iv: "a769af82f34ce2f919cf3bb6eec9d85c"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.855371 28906 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:30.857750 28920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:30.859032 28906 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:44:30.859160 28906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/wal
uuid: "f0ef234bfba9453b8c0d047720c6d16c"
format_stamp: "Formatted at 2026-09-28 06:44:30 on dist-test-slave-gtr8"
server_key: "ad722cde1198c6052b88d29391c0ec4c"
server_key_iv: "a769af82f34ce2f919cf3bb6eec9d85c"
server_key_version: "encryptionkey@0"
I20260928 06:44:30.859263 28906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:30.879004 28906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:30.879830 28906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:30.880102 28906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:30.880928 28906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 06:44:30.882167 28906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 06:44:30.882226 28906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:30.882293 28906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 06:44:30.882321 28906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:30.892460 28906 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.195:44797
I20260928 06:44:30.892525 29034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.195:44797 every 8 connection(s)
I20260928 06:44:30.893340 28906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/data/info.pb
I20260928 06:44:30.895182 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 28906
I20260928 06:44:30.895293 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/wal/instance
I20260928 06:44:30.895572 8799 external_mini_cluster.cc:1428] Setting key 875806f43bb2ec2f01a2f8b9bbeac666
I20260928 06:44:30.903232 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.895213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.195:33845 (local address 127.8.151.254:35027)
0928 06:44:30.895356 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:30.895362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:30.896577 (+ 1215us) server_negotiation.cc:407] Connection header received
0928 06:44:30.898132 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:30.898136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:30.898203 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:30.898301 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:30.899118 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.899995 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.901202 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.901572 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.901684 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.902719 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:30.902724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:30.902727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:30.902743 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:30.902769 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:30.902782 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:30.902866 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:30.902973 (+ 107us) server_negotiation.cc:299] Negotiation successful
0928 06:44:30.903034 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 06:44:30.903995 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.895707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:35027 (local address 127.8.151.195:33845)
0928 06:44:30.896430 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:30.896479 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:30.897911 (+ 1432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:30.898431 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:30.898437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:30.898516 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:30.898981 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.898993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.900113 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.900115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.901096 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.901106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.901697 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.901699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.901843 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.902473 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 06:44:30.902474 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:30.902511 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 06:44:30.902570 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:30.902862 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:30.902867 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:30.902870 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 06:44:30.902955 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 06:44:30.903206 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":139,"threads_started":1}
I20260928 06:44:30.904979 29035 heartbeater.cc:347] Connected to a master server at 127.8.151.254:35027
I20260928 06:44:30.905218 29035 heartbeater.cc:464] Registering TS with master...
I20260928 06:44:30.905797 29035 heartbeater.cc:511] Master 127.8.151.254:35027 requested a full tablet report, sending...
I20260928 06:44:30.906832 28581 ts_manager.cc:194] Registered new tserver with Master: f0ef234bfba9453b8c0d047720c6d16c (127.8.151.195:44797)
I20260928 06:44:30.907399 28581 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.151.195:33845
I20260928 06:44:30.909345 8799 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 06:44:30.914734 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.911211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39780 (local address 127.8.151.254:35027)
0928 06:44:30.911314 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:30.911317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:30.911549 (+ 232us) server_negotiation.cc:407] Connection header received
0928 06:44:30.911649 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:30.911652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:30.911694 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:30.911755 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:30.912020 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.912504 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.913298 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.913520 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.913602 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.914378 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:30.914381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:30.914384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:30.914396 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:30.914413 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:30.914421 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:30.914477 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:30.914560 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 06:44:30.914610 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 06:44:30.917373 28581 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 06:44:30.920197 28581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 06:44:30.934628 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.928411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:43235 (local address 127.0.0.1:44878)
0928 06:44:30.928753 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:30.928804 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:30.928938 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:30.929808 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:30.929814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:30.929837 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:30.929970 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.929977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.931330 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.931333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.932717 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.932729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.934028 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.934031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.934217 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.934254 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 06:44:30.934359 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 06:44:30.934477 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":49,"thread_start_us":85,"threads_started":1}
I20260928 06:44:30.935338 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.928571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44878 (local address 127.8.151.193:43235)
0928 06:44:30.928887 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:30.928894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:30.928921 (+ 27us) server_negotiation.cc:407] Connection header received
0928 06:44:30.929247 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:30.929253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:30.929466 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:30.929623 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:30.930091 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.931207 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.932905 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.933673 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.933772 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.933967 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:30.935048 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0928 06:44:30.935179 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":105,"threads_started":1}
I20260928 06:44:30.936088 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.929396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.194:46505 (local address 127.0.0.1:51352)
0928 06:44:30.930756 (+ 1360us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:30.930768 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:30.930854 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:30.931532 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:30.931535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:30.931551 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:30.931671 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.931677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.933099 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.933103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.934301 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.934310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.935727 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.935730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.935812 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.935835 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:44:30.935902 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 06:44:30.935965 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"mutex_wait_us":87,"thread_start_us":82,"threads_started":1}
I20260928 06:44:30.936996 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.929085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:44797 (local address 127.0.0.1:34290)
0928 06:44:30.929814 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:30.929826 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:30.929895 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:30.930556 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:30.930569 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:30.930591 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:30.930703 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.930711 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.933089 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.933093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.934491 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.934504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.936621 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.936625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.936722 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.936745 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:44:30.936817 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 06:44:30.936878 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":92,"thread_start_us":89,"threads_started":1}
I20260928 06:44:30.937142 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.929246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34290 (local address 127.8.151.195:44797)
0928 06:44:30.930114 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:30.930126 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:30.930149 (+ 23us) server_negotiation.cc:407] Connection header received
0928 06:44:30.930216 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:30.930220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:30.930405 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:30.930545 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:30.930865 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.932942 (+ 2077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.934663 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.936473 (+ 1810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.936614 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.936804 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:30.936859 (+ 55us) server_negotiation.cc:299] Negotiation successful
0928 06:44:30.936955 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":158,"threads_started":1}
I20260928 06:44:30.938251 28704 tablet_service.cc:1467] Processing CreateTablet for tablet 4a0961af6a754b718fb883e2521aac39 (DEFAULT_TABLE table=test-table [id=defad1ac6d774bcfa91f5dd9feef82f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:30.939561 28704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a0961af6a754b718fb883e2521aac39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:30.942039 28968 tablet_service.cc:1467] Processing CreateTablet for tablet 4a0961af6a754b718fb883e2521aac39 (DEFAULT_TABLE table=test-table [id=defad1ac6d774bcfa91f5dd9feef82f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:30.943125 28968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a0961af6a754b718fb883e2521aac39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:30.943188 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.929780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.8.151.194:46505)
0928 06:44:30.930082 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:30.930094 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:30.930941 (+ 847us) server_negotiation.cc:407] Connection header received
0928 06:44:30.931015 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:30.931019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:30.931211 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:30.931392 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:30.931794 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.932986 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.934419 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.935391 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.935487 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.935662 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:30.942841 (+ 7179us) server_negotiation.cc:299] Negotiation successful
0928 06:44:30.942968 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":107,"threads_started":1}
I20260928 06:44:30.946190 28836 tablet_service.cc:1467] Processing CreateTablet for tablet 4a0961af6a754b718fb883e2521aac39 (DEFAULT_TABLE table=test-table [id=defad1ac6d774bcfa91f5dd9feef82f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 06:44:30.947256 28836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a0961af6a754b718fb883e2521aac39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:30.950153 29053 tablet_bootstrap.cc:492] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: Bootstrap starting.
I20260928 06:44:30.950160 29052 tablet_bootstrap.cc:492] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2: Bootstrap starting.
I20260928 06:44:30.952531 29053 tablet_bootstrap.cc:654] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:30.952533 29052 tablet_bootstrap.cc:654] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:30.953433 29052 log.cc:824] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:30.953444 29053 log.cc:824] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:30.955116 29055 tablet_bootstrap.cc:492] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: Bootstrap starting.
I20260928 06:44:30.955785 29052 tablet_bootstrap.cc:492] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2: No bootstrap required, opened a new log
I20260928 06:44:30.955816 29053 tablet_bootstrap.cc:492] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: No bootstrap required, opened a new log
I20260928 06:44:30.955981 29053 ts_tablet_manager.cc:1397] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260928 06:44:30.956013 29052 ts_tablet_manager.cc:1397] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:44:30.957320 29055 tablet_bootstrap.cc:654] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:30.958113 29055 log.cc:824] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:30.960129 29055 tablet_bootstrap.cc:492] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: No bootstrap required, opened a new log
I20260928 06:44:30.960331 29055 ts_tablet_manager.cc:1397] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260928 06:44:30.960358 29053 raft_consensus.cc:348] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 } } peers { permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 } } peers { permanent_uuid: "704059d24f87441c9a6af0b950b3c2d3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 46505 } }
I20260928 06:44:30.960361 29052 raft_consensus.cc:348] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 } } peers { permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 } } peers { permanent_uuid: "704059d24f87441c9a6af0b950b3c2d3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 46505 } }
I20260928 06:44:30.960600 29053 raft_consensus.cc:374] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:30.960641 29053 raft_consensus.cc:729] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0ef234bfba9453b8c0d047720c6d16c, State: Initialized, Role: FOLLOWER
I20260928 06:44:30.960633 29052 raft_consensus.cc:374] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:30.960688 29052 raft_consensus.cc:729] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6859b87ea6054d6fab554cc3a73f9cf2, State: Initialized, Role: FOLLOWER
I20260928 06:44:30.961062 29053 consensus_queue.cc:260] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 } } peers { permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 } } peers { permanent_uuid: "704059d24f87441c9a6af0b950b3c2d3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 46505 } }
I20260928 06:44:30.961153 29052 consensus_queue.cc:260] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 } } peers { permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 } } peers { permanent_uuid: "704059d24f87441c9a6af0b950b3c2d3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 46505 } }
I20260928 06:44:30.961699 29035 heartbeater.cc:503] Master 127.8.151.254:35027 was elected leader, sending a full tablet report...
I20260928 06:44:30.962018 29053 ts_tablet_manager.cc:1428] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:44:30.963240 29052 ts_tablet_manager.cc:1428] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260928 06:44:30.963581 28770 heartbeater.cc:503] Master 127.8.151.254:35027 was elected leader, sending a full tablet report...
I20260928 06:44:30.964429 29055 raft_consensus.cc:348] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 } } peers { permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 } } peers { permanent_uuid: "704059d24f87441c9a6af0b950b3c2d3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 46505 } }
I20260928 06:44:30.964659 29055 raft_consensus.cc:374] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:30.964747 29055 raft_consensus.cc:729] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 704059d24f87441c9a6af0b950b3c2d3, State: Initialized, Role: FOLLOWER
I20260928 06:44:30.965194 29055 consensus_queue.cc:260] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 } } peers { permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 } } peers { permanent_uuid: "704059d24f87441c9a6af0b950b3c2d3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 46505 } }
I20260928 06:44:30.966172 29055 ts_tablet_manager.cc:1428] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260928 06:44:30.966264 28903 heartbeater.cc:503] Master 127.8.151.254:35027 was elected leader, sending a full tablet report...
I20260928 06:44:30.976567 29060 raft_consensus.cc:482] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:44:30.976693 29060 raft_consensus.cc:504] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 } } peers { permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 } } peers { permanent_uuid: "704059d24f87441c9a6af0b950b3c2d3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 46505 } }
I20260928 06:44:30.977644 29060 leader_election.cc:288] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6859b87ea6054d6fab554cc3a73f9cf2 (127.8.151.193:43235), f0ef234bfba9453b8c0d047720c6d16c (127.8.151.195:44797)
I20260928 06:44:30.982052 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.978242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:35089 (local address 127.8.151.195:44797)
0928 06:44:30.978367 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:30.978372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:30.978481 (+ 109us) server_negotiation.cc:407] Connection header received
0928 06:44:30.978593 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:30.978595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:30.978634 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:30.978692 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:30.978931 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.979484 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.981128 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.981566 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.981670 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.981721 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:30.981879 (+ 158us) server_negotiation.cc:299] Negotiation successful
0928 06:44:30.981940 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:44:30.982077 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.978056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.195:44797 (local address 127.8.151.194:35089)
0928 06:44:30.978398 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:30.978408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:30.978515 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:30.978707 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:30.978710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:30.978724 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:30.978840 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.978845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.979581 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.979584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.980989 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.980995 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.981674 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.981678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.981770 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.981793 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 06:44:30.981858 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 06:44:30.981923 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":95,"spinlock_wait_cycles":768,"thread_start_us":107,"threads_started":1}
I20260928 06:44:30.982533 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.977832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:43235 (local address 127.8.151.194:51813)
0928 06:44:30.977932 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:30.977947 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:30.978025 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:30.978423 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:30.978426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:30.978446 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 06:44:30.978608 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.978617 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.979722 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.979725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.980983 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:30.980995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.982118 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.982122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:30.982232 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.982258 (+ 26us) client_negotiation.cc:769] Sending connection context
0928 06:44:30.982336 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 06:44:30.982404 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":40,"spinlock_wait_cycles":4352}
I20260928 06:44:30.982648 28988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a0961af6a754b718fb883e2521aac39" candidate_uuid: "704059d24f87441c9a6af0b950b3c2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0ef234bfba9453b8c0d047720c6d16c" is_pre_election: true
I20260928 06:44:30.982942 28988 raft_consensus.cc:2454] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 704059d24f87441c9a6af0b950b3c2d3 in term 0.
I20260928 06:44:30.982996 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:30.977935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.151.194:51813 (local address 127.8.151.193:43235)
0928 06:44:30.978084 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:30.978089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:30.978106 (+ 17us) server_negotiation.cc:407] Connection header received
0928 06:44:30.978160 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:30.978163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:30.978213 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:30.978296 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 06:44:30.978781 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.979616 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.981125 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:30.981795 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:30.981916 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:30.981967 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:30.982798 (+ 831us) server_negotiation.cc:299] Negotiation successful
0928 06:44:30.982875 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 06:44:30.983407 28788 leader_election.cc:302] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 704059d24f87441c9a6af0b950b3c2d3, f0ef234bfba9453b8c0d047720c6d16c; no voters:
I20260928 06:44:30.983534 28724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a0961af6a754b718fb883e2521aac39" candidate_uuid: "704059d24f87441c9a6af0b950b3c2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" is_pre_election: true
I20260928 06:44:30.983695 29060 raft_consensus.cc:2781] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 06:44:30.983774 29060 raft_consensus.cc:482] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 06:44:30.983811 29060 raft_consensus.cc:3037] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:30.983786 28724 raft_consensus.cc:2454] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 704059d24f87441c9a6af0b950b3c2d3 in term 0.
W20260928 06:44:30.984678 28771 tablet.cc:2369] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:44:30.985056 29060 raft_consensus.cc:504] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 } } peers { permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 } } peers { permanent_uuid: "704059d24f87441c9a6af0b950b3c2d3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 46505 } }
I20260928 06:44:30.985636 29060 leader_election.cc:288] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 6859b87ea6054d6fab554cc3a73f9cf2 (127.8.151.193:43235), f0ef234bfba9453b8c0d047720c6d16c (127.8.151.195:44797)
I20260928 06:44:30.985848 28724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a0961af6a754b718fb883e2521aac39" candidate_uuid: "704059d24f87441c9a6af0b950b3c2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6859b87ea6054d6fab554cc3a73f9cf2"
I20260928 06:44:30.985961 28724 raft_consensus.cc:3037] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:30.986959 28724 raft_consensus.cc:2454] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 704059d24f87441c9a6af0b950b3c2d3 in term 1.
I20260928 06:44:30.987252 28988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a0961af6a754b718fb883e2521aac39" candidate_uuid: "704059d24f87441c9a6af0b950b3c2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0ef234bfba9453b8c0d047720c6d16c"
I20260928 06:44:30.987308 28788 leader_election.cc:302] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6859b87ea6054d6fab554cc3a73f9cf2, 704059d24f87441c9a6af0b950b3c2d3; no voters:
I20260928 06:44:30.987365 28988 raft_consensus.cc:3037] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:30.987474 29060 raft_consensus.cc:2781] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:30.987655 29060 raft_consensus.cc:686] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 1 LEADER]: Becoming Leader. State: Replica: 704059d24f87441c9a6af0b950b3c2d3, State: Running, Role: LEADER
I20260928 06:44:30.988138 29060 consensus_queue.cc:237] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 } } peers { permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 } } peers { permanent_uuid: "704059d24f87441c9a6af0b950b3c2d3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 46505 } }
I20260928 06:44:30.988415 28988 raft_consensus.cc:2454] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 704059d24f87441c9a6af0b950b3c2d3 in term 1.
I20260928 06:44:30.991586 28581 catalog_manager.cc:5495] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 704059d24f87441c9a6af0b950b3c2d3 (127.8.151.194). New cstate: current_term: 1 leader_uuid: "704059d24f87441c9a6af0b950b3c2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "704059d24f87441c9a6af0b950b3c2d3" member_type: VOTER last_known_addr { host: "127.8.151.194" port: 46505 } health_report { overall_health: HEALTHY } } }
I20260928 06:44:31.006800 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:31.002972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51364 (local address 127.8.151.194:46505)
0928 06:44:31.003095 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:31.003098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:31.003114 (+ 16us) server_negotiation.cc:407] Connection header received
0928 06:44:31.003256 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:31.003259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:31.003316 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:31.003408 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:31.003686 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:31.004192 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:31.005376 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:31.005594 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:31.005731 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:31.006006 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:31.006404 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:31.006492 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:31.006571 (+ 79us) server_negotiation.cc:299] Negotiation successful
0928 06:44:31.006652 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 06:44:31.012892 28724 raft_consensus.cc:1261] T 4a0961af6a754b718fb883e2521aac39 P 6859b87ea6054d6fab554cc3a73f9cf2 [term 1 FOLLOWER]: Refusing update from remote peer 704059d24f87441c9a6af0b950b3c2d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:31.012892 28988 raft_consensus.cc:1261] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c [term 1 FOLLOWER]: Refusing update from remote peer 704059d24f87441c9a6af0b950b3c2d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:31.013582 29060 consensus_queue.cc:1035] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0ef234bfba9453b8c0d047720c6d16c" member_type: VOTER last_known_addr { host: "127.8.151.195" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:31.013759 29062 consensus_queue.cc:1035] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6859b87ea6054d6fab554cc3a73f9cf2" member_type: VOTER last_known_addr { host: "127.8.151.193" port: 43235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:31.022953 29067 mvcc.cc:205] Tried to move back new op lower bound from 7334206959660957696 to 7334206959570296832. Current Snapshot: MvccSnapshot[applied={T|T < 7334206959660957696}]
I20260928 06:44:31.029354 28579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:31.029510 28579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 06:44:31.032537 28579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77c3435c9684439da66603c36d15c6ab: Sending DeleteTablet for 3 replicas of tablet 4a0961af6a754b718fb883e2521aac39
I20260928 06:44:31.033375 28968 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a0961af6a754b718fb883e2521aac39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:31 UTC) from {username='slave'} at 127.0.0.1:34290
I20260928 06:44:31.033515 28704 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a0961af6a754b718fb883e2521aac39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:31 UTC) from {username='slave'} at 127.0.0.1:44878
I20260928 06:44:31.033674 28836 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a0961af6a754b718fb883e2521aac39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:31 UTC) from {username='slave'} at 127.0.0.1:51352
I20260928 06:44:31.034011 28835 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a0961af6a754b718fb883e2521aac39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 06:44:31 UTC) from {username='slave'} at 127.0.0.1:51352
I20260928 06:44:31.034073 29077 tablet_replica.cc:330] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: stopping tablet replica
I20260928 06:44:31.034389 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 28642
I20260928 06:44:31.034408 29077 raft_consensus.cc:2229] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 1 LEADER]: Raft consensus shutting down.
I20260928 06:44:31.034817 29077 raft_consensus.cc:2258] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:31.036397 29077 ts_tablet_manager.cc:1905] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:44:31.037308 29075 tablet_replica.cc:330] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: stopping tablet replica
I20260928 06:44:31.037571 29075 raft_consensus.cc:2229] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 06:44:31.037860 29075 raft_consensus.cc:2258] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 06:44:31.039120 29075 ts_tablet_manager.cc:1905] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 06:44:31.039359 29077 ts_tablet_manager.cc:1918] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:44:31.039523 29077 log.cc:1196] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-1/wal/wals/4a0961af6a754b718fb883e2521aac39
I20260928 06:44:31.039907 29077 ts_tablet_manager.cc:1939] T 4a0961af6a754b718fb883e2521aac39 P 704059d24f87441c9a6af0b950b3c2d3: Deleting consensus metadata
I20260928 06:44:31.041172 28565 catalog_manager.cc:4841] TS 704059d24f87441c9a6af0b950b3c2d3 (127.8.151.194:46505): tablet 4a0961af6a754b718fb883e2521aac39 (table test-table [id=defad1ac6d774bcfa91f5dd9feef82f5]) successfully deleted
W20260928 06:44:31.041620 28565 catalog_manager.cc:4808] TS 704059d24f87441c9a6af0b950b3c2d3 (127.8.151.194:46505): delete failed for tablet 4a0961af6a754b718fb883e2521aac39 because the tablet was not found. No further retry: Not found: Tablet not found: 4a0961af6a754b718fb883e2521aac39
I20260928 06:44:31.041729 29075 ts_tablet_manager.cc:1918] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 06:44:31.041837 29075 log.cc:1196] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: Deleting WAL directory at /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790577605970741-8799-0/minicluster-data/ts-2/wal/wals/4a0961af6a754b718fb883e2521aac39
I20260928 06:44:31.042172 29075 ts_tablet_manager.cc:1939] T 4a0961af6a754b718fb883e2521aac39 P f0ef234bfba9453b8c0d047720c6d16c: Deleting consensus metadata
I20260928 06:44:31.043489 28565 catalog_manager.cc:4841] TS f0ef234bfba9453b8c0d047720c6d16c (127.8.151.195:44797): tablet 4a0961af6a754b718fb883e2521aac39 (table test-table [id=defad1ac6d774bcfa91f5dd9feef82f5]) successfully deleted
I20260928 06:44:31.049639 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 28773
W20260928 06:44:31.052384 28565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.151.193:43235 (error 108)
I20260928 06:44:31.053710 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:31.053402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.193:43235 (local address 127.0.0.1:44880)
0928 06:44:31.053542 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:31.053615 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.193:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 06:44:31.053972 28565 catalog_manager.cc:4584] TS 6859b87ea6054d6fab554cc3a73f9cf2 (127.8.151.193:43235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a0961af6a754b718fb883e2521aac39: Network error: Client connection negotiation failed: client connection to 127.8.151.193:43235: connect: Connection refused (error 111)
I20260928 06:44:31.055802 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 28906
I20260928 06:44:31.061583 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 28550
2026-09-28T06:44:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1104 ms)
[----------] 6 tests from EncryptionPolicyTest (10613 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:31 dist-test-slave-gtr8 krb5kdc[29081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:31 dist-test-slave-gtr8 krb5kdc[29081](info): set up 2 sockets
Sep 28 06:44:31 dist-test-slave-gtr8 krb5kdc[29081](info): commencing operation
krb5kdc: starting...
W20260928 06:44:32.051668 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.969s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:32 dist-test-slave-gtr8 krb5kdc[29081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:32.171996 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46129
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:32.297683 29094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:32.297930 29094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:32.297971 29094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:32.301929 29094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:32.301985 29094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:32.302003 29094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:32.302018 29094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:32.302031 29094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:32.307171 29094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46129
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:32.308247 29094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:32.309309 29094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:32.317700 29099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:32.317708 29100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:32.317875 29094 server_base.cc:1034] running on GCE node
W20260928 06:44:32.317724 29102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:32.318629 29094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:32.319773 29094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:32.320963 29094 hybrid_clock.cc:648] HybridClock initialized: now 1790577872320942 us; error 35 us; skew 500 ppm
Sep 28 06:44:32 dist-test-slave-gtr8 krb5kdc[29081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577872, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:32.324122 29094 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:32.325134 29094 webserver.cc:460] Webserver started at http://127.0.0.1:43043/ using document root <none> and password file <none>
I20260928 06:44:32.325773 29094 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:32.325853 29094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:32.326069 29094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:32.328598 29094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "b878f680c6844054a1a8c6c3ec4fa7fe"
format_stamp: "Formatted at 2026-09-28 06:44:32 on dist-test-slave-gtr8"
server_key: "ce2b5f4e99f859e203b7fe886d779313"
server_key_iv: "d62d027b4fae62ca6002c0e29c42ad83"
server_key_version: "encryptionkey@0"
I20260928 06:44:32.329229 29094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "b878f680c6844054a1a8c6c3ec4fa7fe"
format_stamp: "Formatted at 2026-09-28 06:44:32 on dist-test-slave-gtr8"
server_key: "ce2b5f4e99f859e203b7fe886d779313"
server_key_iv: "d62d027b4fae62ca6002c0e29c42ad83"
server_key_version: "encryptionkey@0"
I20260928 06:44:32.333688 29094 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:44:32.336546 29109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:32.337898 29094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 06:44:32.338025 29094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "b878f680c6844054a1a8c6c3ec4fa7fe"
format_stamp: "Formatted at 2026-09-28 06:44:32 on dist-test-slave-gtr8"
server_key: "ce2b5f4e99f859e203b7fe886d779313"
server_key_iv: "d62d027b4fae62ca6002c0e29c42ad83"
server_key_version: "encryptionkey@0"
I20260928 06:44:32.338135 29094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:32.356050 29094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:32.359027 29094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:32.359228 29094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:32.365389 29094 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46129
I20260928 06:44:32.365588 29162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46129 every 8 connection(s)
I20260928 06:44:32.366338 29094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:32.368700 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 29094
I20260928 06:44:32.368837 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:32.369102 8799 external_mini_cluster.cc:1428] Setting key e4017564b3d273c8299dd4a2475db939
I20260928 06:44:32.369632 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:32.376318 29163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe: Bootstrap starting.
I20260928 06:44:32.378851 29163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe: Neither blocks nor log segments found. Creating new log.
Sep 28 06:44:32 dist-test-slave-gtr8 krb5kdc[29081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 06:44:32.379772 29163 log.cc:824] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:32.381824 29163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe: No bootstrap required, opened a new log
I20260928 06:44:32.385293 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:32.370408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40527 (local address 127.0.0.1:46129)
0928 06:44:32.370813 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:32.370832 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:32.370875 (+ 43us) server_negotiation.cc:407] Connection header received
0928 06:44:32.371717 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:32.371741 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:32.372167 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:32.372533 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:32.373367 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:32.374846 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:32.376023 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:32.376713 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:32.376870 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:32.379887 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:32.379893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:32.379910 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:32.379943 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:32.381980 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:32.382549 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:32.382555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:32.382561 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:32.382710 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:32.383106 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:32.383109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:32.383111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:32.383620 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:32.383826 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:32.384158 (+ 332us) server_negotiation.cc:299] Negotiation successful
0928 06:44:32.384377 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":124,"threads_started":1}
I20260928 06:44:32.385896 29163 raft_consensus.cc:348] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b878f680c6844054a1a8c6c3ec4fa7fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46129 } }
I20260928 06:44:32.386121 29163 raft_consensus.cc:374] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:32.386204 29163 raft_consensus.cc:729] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b878f680c6844054a1a8c6c3ec4fa7fe, State: Initialized, Role: FOLLOWER
I20260928 06:44:32.387352 29163 consensus_queue.cc:260] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b878f680c6844054a1a8c6c3ec4fa7fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46129 } }
I20260928 06:44:32.387516 29163 raft_consensus.cc:388] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:32.387595 29163 raft_consensus.cc:482] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:32.387706 29163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:32.388211 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:32.388797 29163 raft_consensus.cc:504] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b878f680c6844054a1a8c6c3ec4fa7fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46129 } }
I20260928 06:44:32.389165 29163 leader_election.cc:302] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b878f680c6844054a1a8c6c3ec4fa7fe; no voters:
I20260928 06:44:32.389537 29163 leader_election.cc:288] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:32.389619 29169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:32.389859 29169 raft_consensus.cc:686] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [term 1 LEADER]: Becoming Leader. State: Replica: b878f680c6844054a1a8c6c3ec4fa7fe, State: Running, Role: LEADER
I20260928 06:44:32.390166 29169 consensus_queue.cc:237] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b878f680c6844054a1a8c6c3ec4fa7fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46129 } }
I20260928 06:44:32.390749 29163 sys_catalog.cc:564] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:32.391696 29173 sys_catalog.cc:455] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b878f680c6844054a1a8c6c3ec4fa7fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b878f680c6844054a1a8c6c3ec4fa7fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46129 } } }
I20260928 06:44:32.391825 29173 sys_catalog.cc:458] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:32.392318 29174 sys_catalog.cc:455] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader b878f680c6844054a1a8c6c3ec4fa7fe. Latest consensus state: current_term: 1 leader_uuid: "b878f680c6844054a1a8c6c3ec4fa7fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b878f680c6844054a1a8c6c3ec4fa7fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46129 } } }
I20260928 06:44:32.392911 29174 sys_catalog.cc:458] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:32.395854 29187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:32.395967 29187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:32.396057 29181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:32.396808 29181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:32.402516 29181 catalog_manager.cc:1347] Generated new cluster ID: d94916b2048d4d57abfe5c57ded9aa90
I20260928 06:44:32.402570 29181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:32.403047 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:32.393418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32989 (local address 127.0.0.1:46129)
0928 06:44:32.393535 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:32.393540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:32.393885 (+ 345us) server_negotiation.cc:407] Connection header received
0928 06:44:32.393995 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:32.393998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:32.394042 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:32.394106 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:32.395029 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:32.395842 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:32.396974 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:32.397332 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:32.397407 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:32.399241 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:32.399246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:32.399249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:32.399276 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:32.400961 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:32.401545 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:32.401549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:32.401551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:32.401632 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:32.402024 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:32.402028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:32.402029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:32.402255 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:32.402375 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:32.402800 (+ 425us) server_negotiation.cc:299] Negotiation successful
0928 06:44:32.402898 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:44:32.420048 29181 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:32.421280 29181 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:32.428061 29181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b878f680c6844054a1a8c6c3ec4fa7fe: Generated new TSK 0
I20260928 06:44:32.428823 29181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:32.433499 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 29094
W20260928 06:44:32.438511 29079 connection.cc:535] client connection to 127.0.0.1:46129 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46129): Connection reset by peer (error 104)
2026-09-28T06:44:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1370 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:32 dist-test-slave-gtr8 krb5kdc[29194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:32 dist-test-slave-gtr8 krb5kdc[29194](info): set up 2 sockets
Sep 28 06:44:32 dist-test-slave-gtr8 krb5kdc[29194](info): commencing operation
krb5kdc: starting...
W20260928 06:44:33.391508 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:33 dist-test-slave-gtr8 krb5kdc[29194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:33.513747 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46059
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:33.624838 29207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:33.625123 29207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:33.625185 29207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:33.629215 29207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:33.629281 29207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:33.629308 29207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:33.629333 29207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:33.629361 29207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:33.634673 29207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46059
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:33.635757 29207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:33.636751 29207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:33.645179 29213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:33.645280 29215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:33.645200 29212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:33.645649 29207 server_base.cc:1034] running on GCE node
I20260928 06:44:33.646206 29207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:33.647372 29207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:33.648536 29207 hybrid_clock.cc:648] HybridClock initialized: now 1790577873648511 us; error 46 us; skew 500 ppm
Sep 28 06:44:33 dist-test-slave-gtr8 krb5kdc[29194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577873, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:33.651304 29207 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:33.652200 29207 webserver.cc:460] Webserver started at http://127.0.0.1:43003/ using document root <none> and password file <none>
I20260928 06:44:33.652736 29207 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:33.652804 29207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:33.652994 29207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:33.655066 29207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "9c4e89d465a4421dbb1299a86ac1254c"
format_stamp: "Formatted at 2026-09-28 06:44:33 on dist-test-slave-gtr8"
server_key: "6d0a9218c84e3b9c8f0af651ee660218"
server_key_iv: "3a21cb6cc3736d2ef4baffad10081060"
server_key_version: "encryptionkey@0"
I20260928 06:44:33.655642 29207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "9c4e89d465a4421dbb1299a86ac1254c"
format_stamp: "Formatted at 2026-09-28 06:44:33 on dist-test-slave-gtr8"
server_key: "6d0a9218c84e3b9c8f0af651ee660218"
server_key_iv: "3a21cb6cc3736d2ef4baffad10081060"
server_key_version: "encryptionkey@0"
I20260928 06:44:33.659406 29207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 06:44:33.661834 29222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:33.663055 29207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:44:33.663200 29207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "9c4e89d465a4421dbb1299a86ac1254c"
format_stamp: "Formatted at 2026-09-28 06:44:33 on dist-test-slave-gtr8"
server_key: "6d0a9218c84e3b9c8f0af651ee660218"
server_key_iv: "3a21cb6cc3736d2ef4baffad10081060"
server_key_version: "encryptionkey@0"
I20260928 06:44:33.663306 29207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:33.679183 29207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:33.682217 29207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:33.682403 29207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:33.689982 29207 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46059
I20260928 06:44:33.690016 29274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46059 every 8 connection(s)
I20260928 06:44:33.690929 29207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:33.693938 29275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:33.699390 29275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c: Bootstrap starting.
I20260928 06:44:33.700696 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 29207
I20260928 06:44:33.700837 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:33.701128 8799 external_mini_cluster.cc:1428] Setting key 4720b832e26411b6a520dc7bc44c2832
I20260928 06:44:33.702050 29275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:33.702886 29275 log.cc:824] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:33.704847 29275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c: No bootstrap required, opened a new log
I20260928 06:44:33.708737 29275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4e89d465a4421dbb1299a86ac1254c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260928 06:44:33.709004 29275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:33.709066 29275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c4e89d465a4421dbb1299a86ac1254c, State: Initialized, Role: FOLLOWER
I20260928 06:44:33.709525 29275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4e89d465a4421dbb1299a86ac1254c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260928 06:44:33.709669 29275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:33.709738 29275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:33.709837 29275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:33.710922 29275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4e89d465a4421dbb1299a86ac1254c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260928 06:44:33.711374 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:33.702280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38897 (local address 127.0.0.1:46059)
0928 06:44:33.702711 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:33.702728 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:33.702771 (+ 43us) server_negotiation.cc:407] Connection header received
0928 06:44:33.703544 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:33.703567 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:33.703998 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:33.704353 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:33.704766 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:33.706128 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:33.707402 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:33.708096 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:33.708249 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:33.709231 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:33.709237 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:33.709253 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:33.709732 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:33.709816 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:33.709845 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:33.709960 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:33.710211 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 06:44:33.710347 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":141,"threads_started":1}
I20260928 06:44:33.711333 29275 leader_election.cc:302] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c4e89d465a4421dbb1299a86ac1254c; no voters:
I20260928 06:44:33.711764 29275 leader_election.cc:288] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:33.711949 29280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:33.712352 29280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [term 1 LEADER]: Becoming Leader. State: Replica: 9c4e89d465a4421dbb1299a86ac1254c, State: Running, Role: LEADER
I20260928 06:44:33.712735 29280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4e89d465a4421dbb1299a86ac1254c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260928 06:44:33.713074 29275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:33.714347 29280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c4e89d465a4421dbb1299a86ac1254c. Latest consensus state: current_term: 1 leader_uuid: "9c4e89d465a4421dbb1299a86ac1254c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4e89d465a4421dbb1299a86ac1254c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } } }
I20260928 06:44:33.714514 29280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:33.714689 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:33.715304 29281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c4e89d465a4421dbb1299a86ac1254c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4e89d465a4421dbb1299a86ac1254c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } } }
I20260928 06:44:33.715478 29281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:33.717993 29295 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:33.718070 29295 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:33.719192 29291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:33.720111 29291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:33.724651 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:33.718705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44387 (local address 127.0.0.1:46059)
0928 06:44:33.718854 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:33.718858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:33.719101 (+ 243us) server_negotiation.cc:407] Connection header received
0928 06:44:33.719285 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:33.719288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:33.719337 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:33.719389 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:33.719831 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:33.720650 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:33.721846 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:33.722188 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:33.722274 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:33.724150 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:33.724154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:33.724157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:33.724170 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:33.724199 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:33.724212 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:33.724268 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:33.724492 (+ 224us) server_negotiation.cc:299] Negotiation successful
0928 06:44:33.724528 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 06:44:33.726763 29291 catalog_manager.cc:1347] Generated new cluster ID: b841002ffd194c80855a6c8c1138a7bf
I20260928 06:44:33.726879 29291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:33.734661 29291 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:33.735957 29291 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:33.746229 29291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c4e89d465a4421dbb1299a86ac1254c: Generated new TSK 0
I20260928 06:44:33.747042 29291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:33.758186 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 29207
W20260928 06:44:33.764261 29192 connection.cc:535] client connection to 127.0.0.1:46059 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46059): Connection reset by peer (error 104)
2026-09-28T06:44:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:33 dist-test-slave-gtr8 krb5kdc[29306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:33 dist-test-slave-gtr8 krb5kdc[29306](info): set up 2 sockets
Sep 28 06:44:33 dist-test-slave-gtr8 krb5kdc[29306](info): commencing operation
krb5kdc: starting...
W20260928 06:44:34.743870 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:34 dist-test-slave-gtr8 krb5kdc[29306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:34.871412 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:34.990926 29319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:34.991317 29319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:34.991386 29319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:34.997282 29319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:34.997359 29319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:34.997386 29319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:34.997409 29319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:34.997438 29319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:35.005215 29319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41757
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:35.006583 29319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:35.007891 29319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:35.017179 29327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:35.017195 29324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:35.017191 29325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:35.017494 29319 server_base.cc:1034] running on GCE node
I20260928 06:44:35.018038 29319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:35.019158 29319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:35.020342 29319 hybrid_clock.cc:648] HybridClock initialized: now 1790577875020323 us; error 28 us; skew 500 ppm
Sep 28 06:44:35 dist-test-slave-gtr8 krb5kdc[29306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577875, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:35.023012 29319 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:35.024003 29319 webserver.cc:460] Webserver started at http://127.0.0.1:38579/ using document root <none> and password file <none>
I20260928 06:44:35.024575 29319 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:35.024641 29319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:35.024832 29319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:35.027015 29319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "284e94e7f3434010b4867e2b0e4c33db"
format_stamp: "Formatted at 2026-09-28 06:44:35 on dist-test-slave-gtr8"
server_key: "9a4c3251ebd26d87ead4df2156395c55"
server_key_iv: "68e9b48eaa97b9cbca0c44a9de67732f"
server_key_version: "encryptionkey@0"
I20260928 06:44:35.027648 29319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "284e94e7f3434010b4867e2b0e4c33db"
format_stamp: "Formatted at 2026-09-28 06:44:35 on dist-test-slave-gtr8"
server_key: "9a4c3251ebd26d87ead4df2156395c55"
server_key_iv: "68e9b48eaa97b9cbca0c44a9de67732f"
server_key_version: "encryptionkey@0"
I20260928 06:44:35.031723 29319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 06:44:35.034790 29334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:35.036250 29319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:35.036373 29319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "284e94e7f3434010b4867e2b0e4c33db"
format_stamp: "Formatted at 2026-09-28 06:44:35 on dist-test-slave-gtr8"
server_key: "9a4c3251ebd26d87ead4df2156395c55"
server_key_iv: "68e9b48eaa97b9cbca0c44a9de67732f"
server_key_version: "encryptionkey@0"
I20260928 06:44:35.036484 29319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:35.042478 29319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:35.045802 29319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:35.045969 29319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:35.053792 29387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41757 every 8 connection(s)
I20260928 06:44:35.053802 29319 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41757
I20260928 06:44:35.054989 29319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:35.057474 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 29319
I20260928 06:44:35.057631 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:35.057942 8799 external_mini_cluster.cc:1428] Setting key b066187bc1f847adc0fef50b7c13767f
I20260928 06:44:35.057951 29388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:35.063634 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:35.059381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47675 (local address 127.0.0.1:41757)
0928 06:44:35.059898 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:35.059918 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:35.059960 (+ 42us) server_negotiation.cc:407] Connection header received
0928 06:44:35.060740 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:35.060762 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:35.061096 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:35.061416 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:35.062179 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:35.062184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:35.062201 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:35.062214 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:35.062294 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:35.062321 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:35.062451 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:35.062522 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 06:44:35.062797 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":185,"threads_started":1}
I20260928 06:44:35.064733 29388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db: Bootstrap starting.
I20260928 06:44:35.067099 29388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:35.067970 29388 log.cc:824] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:35.068778 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:35.069967 29388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db: No bootstrap required, opened a new log
I20260928 06:44:35.072548 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:35.070774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56547 (local address 127.0.0.1:41757)
0928 06:44:35.070883 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:35.070887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:35.071074 (+ 187us) server_negotiation.cc:407] Connection header received
0928 06:44:35.071255 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:35.071258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:35.071293 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:35.071339 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:35.072092 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:35.072096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:35.072098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:35.072141 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:35.072166 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:35.072176 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:35.072248 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:35.072345 (+ 97us) server_negotiation.cc:299] Negotiation successful
0928 06:44:35.072384 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 06:44:35.074112 29388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284e94e7f3434010b4867e2b0e4c33db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41757 } }
I20260928 06:44:35.074340 29388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:35.074378 29388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284e94e7f3434010b4867e2b0e4c33db, State: Initialized, Role: FOLLOWER
I20260928 06:44:35.074863 29388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284e94e7f3434010b4867e2b0e4c33db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41757 } }
I20260928 06:44:35.075032 29388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:35.075098 29388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:35.075222 29388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:35.076303 29388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284e94e7f3434010b4867e2b0e4c33db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41757 } }
I20260928 06:44:35.076681 29388 leader_election.cc:302] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 284e94e7f3434010b4867e2b0e4c33db; no voters:
I20260928 06:44:35.077034 29388 leader_election.cc:288] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:35.077126 29398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:35.077327 29398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [term 1 LEADER]: Becoming Leader. State: Replica: 284e94e7f3434010b4867e2b0e4c33db, State: Running, Role: LEADER
I20260928 06:44:35.077606 29398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284e94e7f3434010b4867e2b0e4c33db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41757 } }
I20260928 06:44:35.078310 29388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:35.079407 29399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "284e94e7f3434010b4867e2b0e4c33db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284e94e7f3434010b4867e2b0e4c33db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41757 } } }
I20260928 06:44:35.079533 29399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:35.079820 29400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 284e94e7f3434010b4867e2b0e4c33db. Latest consensus state: current_term: 1 leader_uuid: "284e94e7f3434010b4867e2b0e4c33db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284e94e7f3434010b4867e2b0e4c33db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41757 } } }
I20260928 06:44:35.079913 29400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:35.079913 29407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:35.083125 29407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:35.091701 29407 catalog_manager.cc:1347] Generated new cluster ID: 342d88af8abc46c39cace04e99228f46
I20260928 06:44:35.091820 29407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:35.115202 29407 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:35.115546 29407 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:35.124604 29407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 284e94e7f3434010b4867e2b0e4c33db: Generated new TSK 0
I20260928 06:44:35.125453 29407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:35.138146 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 29319
2026-09-28T06:44:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:35 dist-test-slave-gtr8 krb5kdc[29419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:35 dist-test-slave-gtr8 krb5kdc[29419](info): set up 2 sockets
Sep 28 06:44:35 dist-test-slave-gtr8 krb5kdc[29419](info): commencing operation
krb5kdc: starting...
W20260928 06:44:36.115634 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:36 dist-test-slave-gtr8 krb5kdc[29419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:36.234498 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41425
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:36.361246 29432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:36.361521 29432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:36.361560 29432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:36.365540 29432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:36.365597 29432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:36.365615 29432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:36.365630 29432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:36.365644 29432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:36.370785 29432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:36.371848 29432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:36.372848 29432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:36.380998 29437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:36.381078 29432 server_base.cc:1034] running on GCE node
W20260928 06:44:36.380995 29440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:36.381264 29438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:36.381819 29432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:36.382864 29432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:36.384023 29432 hybrid_clock.cc:648] HybridClock initialized: now 1790577876383987 us; error 53 us; skew 500 ppm
Sep 28 06:44:36 dist-test-slave-gtr8 krb5kdc[29419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577876, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:36.391016 29432 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:36.391961 29432 webserver.cc:460] Webserver started at http://127.0.0.1:38197/ using document root <none> and password file <none>
I20260928 06:44:36.392488 29432 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:36.392547 29432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:36.392733 29432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:36.394724 29432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "a7fdcf60fb364ac083d465314cc3f0f3"
format_stamp: "Formatted at 2026-09-28 06:44:36 on dist-test-slave-gtr8"
server_key: "42b59abe78ac204a30ba03ca273289e8"
server_key_iv: "fd28f08e0e90c03339d4123aed05e383"
server_key_version: "encryptionkey@0"
I20260928 06:44:36.395283 29432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "a7fdcf60fb364ac083d465314cc3f0f3"
format_stamp: "Formatted at 2026-09-28 06:44:36 on dist-test-slave-gtr8"
server_key: "42b59abe78ac204a30ba03ca273289e8"
server_key_iv: "fd28f08e0e90c03339d4123aed05e383"
server_key_version: "encryptionkey@0"
I20260928 06:44:36.398886 29432 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:44:36.401683 29448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:36.402930 29432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260928 06:44:36.403041 29432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "a7fdcf60fb364ac083d465314cc3f0f3"
format_stamp: "Formatted at 2026-09-28 06:44:36 on dist-test-slave-gtr8"
server_key: "42b59abe78ac204a30ba03ca273289e8"
server_key_iv: "fd28f08e0e90c03339d4123aed05e383"
server_key_version: "encryptionkey@0"
I20260928 06:44:36.403155 29432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:36.420063 29432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:36.423338 29432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:36.423512 29432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:36.430488 29432 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41425
I20260928 06:44:36.430533 29500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41425 every 8 connection(s)
I20260928 06:44:36.431504 29432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:36.434239 29501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:36.439585 29501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3: Bootstrap starting.
I20260928 06:44:36.441447 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 29432
I20260928 06:44:36.441596 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:36.441875 8799 external_mini_cluster.cc:1428] Setting key 689fb09452860a601a9029e00d18a3c2
I20260928 06:44:36.442205 29501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:36.443073 29501 log.cc:824] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:36.445503 29501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3: No bootstrap required, opened a new log
I20260928 06:44:36.449513 29501 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fdcf60fb364ac083d465314cc3f0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41425 } }
I20260928 06:44:36.449755 29501 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:36.449846 29501 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7fdcf60fb364ac083d465314cc3f0f3, State: Initialized, Role: FOLLOWER
I20260928 06:44:36.450302 29501 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fdcf60fb364ac083d465314cc3f0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41425 } }
I20260928 06:44:36.450434 29501 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:36.450531 29501 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:36.450629 29501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:36.451627 29501 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fdcf60fb364ac083d465314cc3f0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41425 } }
Sep 28 06:44:36 dist-test-slave-gtr8 krb5kdc[29419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 06:44:36.451970 29501 leader_election.cc:302] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7fdcf60fb364ac083d465314cc3f0f3; no voters:
I20260928 06:44:36.452277 29501 leader_election.cc:288] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:36.452358 29506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:36.452574 29506 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [term 1 LEADER]: Becoming Leader. State: Replica: a7fdcf60fb364ac083d465314cc3f0f3, State: Running, Role: LEADER
I20260928 06:44:36.453163 29506 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fdcf60fb364ac083d465314cc3f0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41425 } }
I20260928 06:44:36.453580 29501 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:36.457943 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:36.443025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40451 (local address 127.0.0.1:41425)
0928 06:44:36.443514 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:36.443533 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:36.443575 (+ 42us) server_negotiation.cc:407] Connection header received
0928 06:44:36.444336 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:36.444359 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:36.444778 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:36.445127 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:36.446022 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:36.447522 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:36.448636 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:36.449329 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:36.449480 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:36.452435 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:36.452441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:36.452459 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:36.452488 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:36.454360 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:36.454858 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:36.454864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:36.454870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:36.455003 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:36.455321 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:36.455324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:36.455325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:36.455797 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:36.455992 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:36.456705 (+ 713us) server_negotiation.cc:299] Negotiation successful
0928 06:44:36.456987 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":132,"threads_started":1}
I20260928 06:44:36.459532 29508 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7fdcf60fb364ac083d465314cc3f0f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fdcf60fb364ac083d465314cc3f0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41425 } } }
I20260928 06:44:36.459651 29508 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:36.459848 29507 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7fdcf60fb364ac083d465314cc3f0f3. Latest consensus state: current_term: 1 leader_uuid: "a7fdcf60fb364ac083d465314cc3f0f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7fdcf60fb364ac083d465314cc3f0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41425 } } }
I20260928 06:44:36.459996 29507 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3 [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:36.458871 29520 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:36.460161 29520 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:36.460309 29523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:36.460911 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:36.461161 29523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:36.466830 29523 catalog_manager.cc:1347] Generated new cluster ID: 022d510ba06140e28aee3e5c3cac2ab2
I20260928 06:44:36.466879 29523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:36.473135 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:36.464653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49451 (local address 127.0.0.1:41425)
0928 06:44:36.464764 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:36.464768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:36.465224 (+ 456us) server_negotiation.cc:407] Connection header received
0928 06:44:36.465316 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:36.465319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:36.465363 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:36.465428 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:36.466281 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:36.466987 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:36.468211 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:36.468556 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:36.468615 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:36.470124 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:36.470128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:36.470130 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:36.470153 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:36.471576 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:36.472004 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:36.472007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:36.472009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:36.472076 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:36.472316 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:36.472319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:36.472321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:36.472489 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:36.472572 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:36.472884 (+ 312us) server_negotiation.cc:299] Negotiation successful
0928 06:44:36.472984 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:44:36.501355 29523 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:36.502445 29523 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:36.512497 29523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7fdcf60fb364ac083d465314cc3f0f3: Generated new TSK 0
I20260928 06:44:36.513195 29523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:36.517946 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 29432
2026-09-28T06:44:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:36 dist-test-slave-gtr8 krb5kdc[29532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:36 dist-test-slave-gtr8 krb5kdc[29532](info): set up 2 sockets
Sep 28 06:44:36 dist-test-slave-gtr8 krb5kdc[29532](info): commencing operation
krb5kdc: starting...
W20260928 06:44:37.491643 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:37 dist-test-slave-gtr8 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:37.618135 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34665
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:37.741608 29545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:37.741875 29545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:37.741916 29545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:37.745985 29545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:37.746048 29545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:37.746066 29545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:37.746081 29545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:37.746095 29545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:37.751401 29545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:37.752434 29545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:37.753420 29545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:37.761547 29550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:37.761547 29553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:37.761564 29551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:37.762048 29545 server_base.cc:1034] running on GCE node
I20260928 06:44:37.762549 29545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:37.763638 29545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:37.764808 29545 hybrid_clock.cc:648] HybridClock initialized: now 1790577877764784 us; error 43 us; skew 500 ppm
Sep 28 06:44:37 dist-test-slave-gtr8 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577877, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:37.767432 29545 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:37.768291 29545 webserver.cc:460] Webserver started at http://127.0.0.1:37199/ using document root <none> and password file <none>
I20260928 06:44:37.768812 29545 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:37.768877 29545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:37.769058 29545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:37.771040 29545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "1be9437156ec4a3a9b4028621194b0aa"
format_stamp: "Formatted at 2026-09-28 06:44:37 on dist-test-slave-gtr8"
server_key: "ea99e07ca4dc0d585ad4afafca94c0d8"
server_key_iv: "cd0d402e0cbc798473ebff4625b7eaba"
server_key_version: "encryptionkey@0"
I20260928 06:44:37.771582 29545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "1be9437156ec4a3a9b4028621194b0aa"
format_stamp: "Formatted at 2026-09-28 06:44:37 on dist-test-slave-gtr8"
server_key: "ea99e07ca4dc0d585ad4afafca94c0d8"
server_key_iv: "cd0d402e0cbc798473ebff4625b7eaba"
server_key_version: "encryptionkey@0"
I20260928 06:44:37.776125 29545 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 06:44:37.778637 29560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:37.779968 29545 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:44:37.780081 29545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "1be9437156ec4a3a9b4028621194b0aa"
format_stamp: "Formatted at 2026-09-28 06:44:37 on dist-test-slave-gtr8"
server_key: "ea99e07ca4dc0d585ad4afafca94c0d8"
server_key_iv: "cd0d402e0cbc798473ebff4625b7eaba"
server_key_version: "encryptionkey@0"
I20260928 06:44:37.780180 29545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:37.796082 29545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:37.799099 29545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:37.799312 29545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:37.806015 29545 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34665
I20260928 06:44:37.806056 29613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34665 every 8 connection(s)
I20260928 06:44:37.806942 29545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:37.810248 29614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:37.814670 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 29545
I20260928 06:44:37.814816 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:37.815078 8799 external_mini_cluster.cc:1428] Setting key c0b3ca568ef6277270fe8585e0beeaf2
I20260928 06:44:37.817364 29614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa: Bootstrap starting.
I20260928 06:44:37.820107 29614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:37.820988 29614 log.cc:824] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:37.823454 29614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa: No bootstrap required, opened a new log
I20260928 06:44:37.823704 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:37.816450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56059 (local address 127.0.0.1:34665)
0928 06:44:37.816874 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:37.816900 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:37.816936 (+ 36us) server_negotiation.cc:407] Connection header received
0928 06:44:37.817573 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:37.817591 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:37.817946 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:37.818246 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:37.818640 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:37.819667 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:37.820579 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:37.821034 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:37.821626 (+ 592us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:37.822047 (+ 421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:37.822052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:37.822064 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:37.822425 (+ 361us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:37.822476 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:37.822497 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:37.822580 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:37.822723 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 06:44:37.822944 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":132,"threads_started":1}
I20260928 06:44:37.826737 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:37.827339 29614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9437156ec4a3a9b4028621194b0aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } }
I20260928 06:44:37.827553 29614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:37.827622 29614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1be9437156ec4a3a9b4028621194b0aa, State: Initialized, Role: FOLLOWER
I20260928 06:44:37.828069 29614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9437156ec4a3a9b4028621194b0aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } }
I20260928 06:44:37.828227 29614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:37.828332 29614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:37.828439 29614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:37.829278 29614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9437156ec4a3a9b4028621194b0aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } }
I20260928 06:44:37.829593 29614 leader_election.cc:302] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1be9437156ec4a3a9b4028621194b0aa; no voters:
I20260928 06:44:37.829901 29614 leader_election.cc:288] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:37.830276 29625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:37.830523 29625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [term 1 LEADER]: Becoming Leader. State: Replica: 1be9437156ec4a3a9b4028621194b0aa, State: Running, Role: LEADER
I20260928 06:44:37.830891 29625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9437156ec4a3a9b4028621194b0aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } }
I20260928 06:44:37.831245 29614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:37.832194 29625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1be9437156ec4a3a9b4028621194b0aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9437156ec4a3a9b4028621194b0aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } } }
I20260928 06:44:37.832361 29625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:37.832584 29625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1be9437156ec4a3a9b4028621194b0aa. Latest consensus state: current_term: 1 leader_uuid: "1be9437156ec4a3a9b4028621194b0aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9437156ec4a3a9b4028621194b0aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } } }
I20260928 06:44:37.832705 29625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:37.834895 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:37.829782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43695 (local address 127.0.0.1:34665)
0928 06:44:37.829914 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:37.829918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:37.830316 (+ 398us) server_negotiation.cc:407] Connection header received
0928 06:44:37.830492 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:37.830495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:37.830540 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:37.830586 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:37.830974 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:37.831733 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:37.832883 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:37.833199 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:37.833289 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:37.834217 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:37.834220 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:37.834223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:37.834235 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:37.834272 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:37.834286 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:37.834345 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:37.834717 (+ 372us) server_negotiation.cc:299] Negotiation successful
0928 06:44:37.834771 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260928 06:44:37.836499 29639 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:37.836565 29639 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:37.836616 29633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:37.837368 29633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:37.842509 29633 catalog_manager.cc:1347] Generated new cluster ID: 313fea0b7d94454fada516d16142c73e
I20260928 06:44:37.842566 29633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:37.854708 29633 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:37.855876 29633 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:37.864243 29633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1be9437156ec4a3a9b4028621194b0aa: Generated new TSK 0
I20260928 06:44:37.864914 29633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:37.871479 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 29545
2026-09-28T06:44:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:37 dist-test-slave-gtr8 krb5kdc[29645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:37 dist-test-slave-gtr8 krb5kdc[29645](info): set up 2 sockets
Sep 28 06:44:37 dist-test-slave-gtr8 krb5kdc[29645](info): commencing operation
krb5kdc: starting...
W20260928 06:44:38.859614 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:38 dist-test-slave-gtr8 krb5kdc[29645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:38.988399 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:39.096244 29658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:39.096511 29658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:39.096555 29658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:39.100610 29658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:39.100678 29658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:39.100698 29658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:39.100713 29658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:39.100726 29658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:39.106112 29658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:39.107221 29658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:39.108301 29658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:39.117008 29664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:39.116993 29666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:39.116999 29663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:39.117465 29658 server_base.cc:1034] running on GCE node
I20260928 06:44:39.118098 29658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:39.119473 29658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:39.120664 29658 hybrid_clock.cc:648] HybridClock initialized: now 1790577879120638 us; error 36 us; skew 500 ppm
Sep 28 06:44:39 dist-test-slave-gtr8 krb5kdc[29645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577879, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:39.123991 29658 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:39.125090 29658 webserver.cc:460] Webserver started at http://127.0.0.1:44425/ using document root <none> and password file <none>
I20260928 06:44:39.125751 29658 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:39.125833 29658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:39.126057 29658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:39.128695 29658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "1ddedf82ac7d42279b577dc02fb1b01a"
format_stamp: "Formatted at 2026-09-28 06:44:39 on dist-test-slave-gtr8"
server_key: "b1a44da46e920f8e1918c364e29d5451"
server_key_iv: "a580e3d506bafa1504ecbb416b98bf64"
server_key_version: "encryptionkey@0"
I20260928 06:44:39.129338 29658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "1ddedf82ac7d42279b577dc02fb1b01a"
format_stamp: "Formatted at 2026-09-28 06:44:39 on dist-test-slave-gtr8"
server_key: "b1a44da46e920f8e1918c364e29d5451"
server_key_iv: "a580e3d506bafa1504ecbb416b98bf64"
server_key_version: "encryptionkey@0"
I20260928 06:44:39.134286 29658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260928 06:44:39.137203 29673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:39.138602 29658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260928 06:44:39.138729 29658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "1ddedf82ac7d42279b577dc02fb1b01a"
format_stamp: "Formatted at 2026-09-28 06:44:39 on dist-test-slave-gtr8"
server_key: "b1a44da46e920f8e1918c364e29d5451"
server_key_iv: "a580e3d506bafa1504ecbb416b98bf64"
server_key_version: "encryptionkey@0"
I20260928 06:44:39.138844 29658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:39.144924 29658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:39.148281 29658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:39.148460 29658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:39.156313 29658 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34659
I20260928 06:44:39.156320 29726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34659 every 8 connection(s)
I20260928 06:44:39.157447 29658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:39.160502 29727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:39.164537 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 29658
I20260928 06:44:39.164688 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:39.164978 8799 external_mini_cluster.cc:1428] Setting key 9b8e678e44b825a43332e94ec8b77e7b
I20260928 06:44:39.167205 29727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a: Bootstrap starting.
I20260928 06:44:39.169713 29727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:39.170459 29727 log.cc:824] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:39.170573 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:39.166348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43911 (local address 127.0.0.1:34659)
0928 06:44:39.166789 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:39.166808 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:39.166850 (+ 42us) server_negotiation.cc:407] Connection header received
0928 06:44:39.167653 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:39.167677 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:39.168050 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:39.168400 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:39.169173 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:39.169179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:39.169195 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:39.169206 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:39.169271 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:39.169296 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:39.169399 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:39.169525 (+ 126us) server_negotiation.cc:299] Negotiation successful
0928 06:44:39.169673 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":116,"threads_started":1}
I20260928 06:44:39.172711 29727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a: No bootstrap required, opened a new log
I20260928 06:44:39.173372 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:39.176811 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:39.175283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54867 (local address 127.0.0.1:34659)
0928 06:44:39.175403 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:39.175407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:39.175620 (+ 213us) server_negotiation.cc:407] Connection header received
0928 06:44:39.175720 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:39.175723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:39.175755 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:39.175817 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:39.176458 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:39.176461 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:39.176464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:39.176475 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:39.176502 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:39.176512 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:39.176579 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:39.176656 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 06:44:39.176693 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 06:44:39.177042 29727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddedf82ac7d42279b577dc02fb1b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } }
I20260928 06:44:39.177311 29727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:39.177402 29727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ddedf82ac7d42279b577dc02fb1b01a, State: Initialized, Role: FOLLOWER
I20260928 06:44:39.177910 29727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddedf82ac7d42279b577dc02fb1b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } }
I20260928 06:44:39.178107 29737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:44:39.178274 29737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddedf82ac7d42279b577dc02fb1b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } }
I20260928 06:44:39.178074 29727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:39.178642 29727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:39.178622 29737 leader_election.cc:302] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ddedf82ac7d42279b577dc02fb1b01a; no voters:
I20260928 06:44:39.178746 29727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:39.178997 29737 leader_election.cc:288] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260928 06:44:39.179893 29727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddedf82ac7d42279b577dc02fb1b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } }
I20260928 06:44:39.180032 29727 leader_election.cc:302] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ddedf82ac7d42279b577dc02fb1b01a; no voters:
I20260928 06:44:39.180111 29727 leader_election.cc:288] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:39.180104 29737 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260928 06:44:39.180836 29737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddedf82ac7d42279b577dc02fb1b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } } }
I20260928 06:44:39.180943 29737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:44:39.181046 29738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:39.181141 29738 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [term 1 LEADER]: Becoming Leader. State: Replica: 1ddedf82ac7d42279b577dc02fb1b01a, State: Running, Role: LEADER
I20260928 06:44:39.181358 29727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:39.181445 29738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddedf82ac7d42279b577dc02fb1b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } }
I20260928 06:44:39.182863 29737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ddedf82ac7d42279b577dc02fb1b01a. Latest consensus state: current_term: 1 leader_uuid: "1ddedf82ac7d42279b577dc02fb1b01a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ddedf82ac7d42279b577dc02fb1b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } } }
I20260928 06:44:39.183022 29737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:39.183449 29752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:39.186129 29752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:39.192329 29752 catalog_manager.cc:1347] Generated new cluster ID: 1bd628cafe864d4fa11275b4846700f5
I20260928 06:44:39.192404 29752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:39.208631 29752 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:39.208952 29752 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:39.220952 29752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ddedf82ac7d42279b577dc02fb1b01a: Generated new TSK 0
I20260928 06:44:39.221601 29752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:39.228188 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 29658
2026-09-28T06:44:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:39 dist-test-slave-gtr8 krb5kdc[29758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:39 dist-test-slave-gtr8 krb5kdc[29758](info): set up 2 sockets
Sep 28 06:44:39 dist-test-slave-gtr8 krb5kdc[29758](info): commencing operation
krb5kdc: starting...
W20260928 06:44:40.207844 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:40 dist-test-slave-gtr8 krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:40.331038 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43271
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:40.444751 29771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:40.445007 29771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:40.445047 29771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:40.449081 29771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:40.449144 29771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:40.449162 29771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:40.449177 29771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:40.449193 29771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:40.454603 29771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:40.455755 29771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:40.456791 29771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:40.465274 29777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:40.465250 29776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:40.465346 29771 server_base.cc:1034] running on GCE node
W20260928 06:44:40.465502 29779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:40.466234 29771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:40.467656 29771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:40.468845 29771 hybrid_clock.cc:648] HybridClock initialized: now 1790577880468821 us; error 38 us; skew 500 ppm
Sep 28 06:44:40 dist-test-slave-gtr8 krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577880, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:40.472218 29771 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:40.473114 29771 webserver.cc:460] Webserver started at http://127.0.0.1:43411/ using document root <none> and password file <none>
I20260928 06:44:40.473655 29771 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:40.473714 29771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:40.473905 29771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:40.475997 29771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "9ab4ad5743294f2ab0c825a18b0a6121"
format_stamp: "Formatted at 2026-09-28 06:44:40 on dist-test-slave-gtr8"
server_key: "458c20f09810c330bdba39f5b6d05f6a"
server_key_iv: "aa5f941f1fc0fada89aff983baed9001"
server_key_version: "encryptionkey@0"
I20260928 06:44:40.476529 29771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "9ab4ad5743294f2ab0c825a18b0a6121"
format_stamp: "Formatted at 2026-09-28 06:44:40 on dist-test-slave-gtr8"
server_key: "458c20f09810c330bdba39f5b6d05f6a"
server_key_iv: "aa5f941f1fc0fada89aff983baed9001"
server_key_version: "encryptionkey@0"
I20260928 06:44:40.480340 29771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260928 06:44:40.482717 29786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:40.483961 29771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:44:40.484072 29771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "9ab4ad5743294f2ab0c825a18b0a6121"
format_stamp: "Formatted at 2026-09-28 06:44:40 on dist-test-slave-gtr8"
server_key: "458c20f09810c330bdba39f5b6d05f6a"
server_key_iv: "aa5f941f1fc0fada89aff983baed9001"
server_key_version: "encryptionkey@0"
I20260928 06:44:40.484166 29771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:40.497630 29771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:40.500576 29771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:40.500742 29771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:40.507771 29771 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43271
I20260928 06:44:40.507805 29838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43271 every 8 connection(s)
I20260928 06:44:40.508682 29771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:40.511370 29839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:40.516647 29839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121: Bootstrap starting.
I20260928 06:44:40.517434 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 29771
I20260928 06:44:40.517572 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:40.517865 8799 external_mini_cluster.cc:1428] Setting key 6fa60adab23ae91a979013df9cfa7540
I20260928 06:44:40.519443 29839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:40.520341 29839 log.cc:824] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:40.522581 29839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121: No bootstrap required, opened a new log
I20260928 06:44:40.526686 29839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab4ad5743294f2ab0c825a18b0a6121" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43271 } }
I20260928 06:44:40.526937 29839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:40.527035 29839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab4ad5743294f2ab0c825a18b0a6121, State: Initialized, Role: FOLLOWER
Sep 28 06:44:40 dist-test-slave-gtr8 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 06:44:40.527514 29839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab4ad5743294f2ab0c825a18b0a6121" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43271 } }
I20260928 06:44:40.527647 29839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:40.527727 29839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:40.527825 29839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:40.528932 29839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab4ad5743294f2ab0c825a18b0a6121" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43271 } }
I20260928 06:44:40.529302 29839 leader_election.cc:302] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ab4ad5743294f2ab0c825a18b0a6121; no voters:
I20260928 06:44:40.529659 29839 leader_election.cc:288] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:40.529771 29844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:40.530072 29844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [term 1 LEADER]: Becoming Leader. State: Replica: 9ab4ad5743294f2ab0c825a18b0a6121, State: Running, Role: LEADER
I20260928 06:44:40.530405 29844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab4ad5743294f2ab0c825a18b0a6121" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43271 } }
I20260928 06:44:40.531071 29839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:40.531714 29844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ab4ad5743294f2ab0c825a18b0a6121. Latest consensus state: current_term: 1 leader_uuid: "9ab4ad5743294f2ab0c825a18b0a6121" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab4ad5743294f2ab0c825a18b0a6121" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43271 } } }
I20260928 06:44:40.531714 29845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ab4ad5743294f2ab0c825a18b0a6121" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab4ad5743294f2ab0c825a18b0a6121" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43271 } } }
I20260928 06:44:40.531925 29845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:40.531950 29844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:40.532595 29852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:40.534092 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:40.519081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:39899 (local address 127.0.0.1:43271)
0928 06:44:40.519534 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:40.519554 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:40.519596 (+ 42us) server_negotiation.cc:407] Connection header received
0928 06:44:40.520241 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:40.520257 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:40.520663 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:40.521000 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:40.521787 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:40.523279 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:40.524429 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:40.525124 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:40.525293 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:40.528230 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:40.528236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:40.528253 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:40.528286 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:40.530303 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:40.530946 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:40.530951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:40.530957 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:40.531096 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:40.531555 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:40.531558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:40.531560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:40.532070 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:40.532341 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:40.532942 (+ 601us) server_negotiation.cc:299] Negotiation successful
0928 06:44:40.533157 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":126,"threads_started":1}
I20260928 06:44:40.535857 29852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:40.537322 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:40.541106 29861 mvcc.cc:205] Tried to move back new op lower bound from 7334206998690615296 to 7334206998653587456. Current Snapshot: MvccSnapshot[applied={T|T < 7334206998690615296}]
I20260928 06:44:40.544826 29852 catalog_manager.cc:1347] Generated new cluster ID: c4e2edfbb970426eae038e02135a789d
I20260928 06:44:40.544884 29852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:40.550307 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:40.541399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:37721 (local address 127.0.0.1:43271)
0928 06:44:40.541495 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:40.541499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:40.541753 (+ 254us) server_negotiation.cc:407] Connection header received
0928 06:44:40.541928 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:40.541931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:40.541974 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:40.542018 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:40.542860 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:40.543603 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:40.544670 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:40.544998 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:40.545066 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:40.546771 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:40.546774 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:40.546776 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:40.546812 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:40.548371 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:40.548910 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:40.548913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:40.548915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:40.548992 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:40.549346 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:40.549349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:40.549351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:40.549542 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:40.549661 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:40.550071 (+ 410us) server_negotiation.cc:299] Negotiation successful
0928 06:44:40.550162 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 06:44:40.558198 29852 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:40.559497 29852 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:40.570665 29852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ab4ad5743294f2ab0c825a18b0a6121: Generated new TSK 0
I20260928 06:44:40.571578 29852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:40.578691 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 29771
W20260928 06:44:40.583609 29756 connection.cc:535] client connection to 127.0.0.1:43271 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43271): Connection reset by peer (error 104)
2026-09-28T06:44:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:40 dist-test-slave-gtr8 krb5kdc[29870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:40 dist-test-slave-gtr8 krb5kdc[29870](info): set up 2 sockets
Sep 28 06:44:40 dist-test-slave-gtr8 krb5kdc[29870](info): commencing operation
krb5kdc: starting...
W20260928 06:44:41.581463 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:41 dist-test-slave-gtr8 krb5kdc[29870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:41.712040 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:41.828817 29883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:41.829196 29883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:41.829257 29883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:41.834293 29883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:41.834371 29883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:41.834396 29883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:41.834416 29883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:41.834447 29883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:41.841354 29883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:41.842732 29883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:41.844122 29883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:41.853606 29891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:41.853829 29888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:41.853757 29883 server_base.cc:1034] running on GCE node
W20260928 06:44:41.853643 29889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:41.854629 29883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:41.855928 29883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:41.857118 29883 hybrid_clock.cc:648] HybridClock initialized: now 1790577881857093 us; error 37 us; skew 500 ppm
Sep 28 06:44:41 dist-test-slave-gtr8 krb5kdc[29870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:41.859963 29883 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:41.861114 29883 webserver.cc:460] Webserver started at http://127.0.0.1:42271/ using document root <none> and password file <none>
I20260928 06:44:41.861783 29883 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:41.861876 29883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:41.862123 29883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:41.864748 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "4a928a92552a4e43ba96973316bdf77a"
format_stamp: "Formatted at 2026-09-28 06:44:41 on dist-test-slave-gtr8"
server_key: "76ec4f7a9923aaa921d0a8c50a579c22"
server_key_iv: "f69f8eb72f0710aaaedb9552d714ea8d"
server_key_version: "encryptionkey@0"
I20260928 06:44:41.865402 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "4a928a92552a4e43ba96973316bdf77a"
format_stamp: "Formatted at 2026-09-28 06:44:41 on dist-test-slave-gtr8"
server_key: "76ec4f7a9923aaa921d0a8c50a579c22"
server_key_iv: "f69f8eb72f0710aaaedb9552d714ea8d"
server_key_version: "encryptionkey@0"
I20260928 06:44:41.869942 29883 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260928 06:44:41.872818 29898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:41.874192 29883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260928 06:44:41.874327 29883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "4a928a92552a4e43ba96973316bdf77a"
format_stamp: "Formatted at 2026-09-28 06:44:41 on dist-test-slave-gtr8"
server_key: "76ec4f7a9923aaa921d0a8c50a579c22"
server_key_iv: "f69f8eb72f0710aaaedb9552d714ea8d"
server_key_version: "encryptionkey@0"
I20260928 06:44:41.874459 29883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:41.895077 29883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:41.898288 29883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:41.898466 29883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:41.905627 29883 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43761
I20260928 06:44:41.905661 29950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43761 every 8 connection(s)
I20260928 06:44:41.906600 29883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:41.908106 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 29883
I20260928 06:44:41.908232 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:41.908510 8799 external_mini_cluster.cc:1428] Setting key 5cc66550b30980830bfa82ef207db608
I20260928 06:44:41.909690 29951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:41.917763 29951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a: Bootstrap starting.
I20260928 06:44:41.918639 29954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:41.909922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:59917 (local address 127.0.0.1:43761)
0928 06:44:41.910422 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:41.910438 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:41.911568 (+ 1130us) server_negotiation.cc:407] Connection header received
0928 06:44:41.912324 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:41.912343 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:41.912697 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:41.913009 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:41.913411 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:41.914437 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:41.915392 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:41.915860 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:41.916009 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:41.916850 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:41.916855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:41.916867 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:41.917243 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:41.917308 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:41.917331 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:41.917440 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:41.917776 (+ 336us) server_negotiation.cc:299] Negotiation successful
0928 06:44:41.917925 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":144,"threads_started":1}
I20260928 06:44:41.920785 29951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:41.921366 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:41.921693 29951 log.cc:824] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:41.924048 29951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a: No bootstrap required, opened a new log
I20260928 06:44:41.928050 29954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:41.923480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:48249 (local address 127.0.0.1:43761)
0928 06:44:41.923578 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:41.923582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:41.923776 (+ 194us) server_negotiation.cc:407] Connection header received
0928 06:44:41.923853 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:41.923856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:41.923901 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:41.923980 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:41.924285 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:41.925003 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:41.926178 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:41.926516 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:41.926630 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:41.927583 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:41.927587 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:41.927590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:41.927603 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:41.927634 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:41.927647 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:41.927739 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:41.927879 (+ 140us) server_negotiation.cc:299] Negotiation successful
0928 06:44:41.927915 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 06:44:41.928550 29951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a928a92552a4e43ba96973316bdf77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260928 06:44:41.928797 29951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:41.928891 29951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a928a92552a4e43ba96973316bdf77a, State: Initialized, Role: FOLLOWER
I20260928 06:44:41.929360 29951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a928a92552a4e43ba96973316bdf77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260928 06:44:41.929497 29951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:41.929562 29951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:41.929657 29951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:41.930583 29951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a928a92552a4e43ba96973316bdf77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260928 06:44:41.930976 29951 leader_election.cc:302] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a928a92552a4e43ba96973316bdf77a; no voters:
I20260928 06:44:41.931349 29951 leader_election.cc:288] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:41.931684 29963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:41.931977 29963 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [term 1 LEADER]: Becoming Leader. State: Replica: 4a928a92552a4e43ba96973316bdf77a, State: Running, Role: LEADER
I20260928 06:44:41.932320 29963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a928a92552a4e43ba96973316bdf77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260928 06:44:41.932659 29951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:41.933948 29962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a928a92552a4e43ba96973316bdf77a. Latest consensus state: current_term: 1 leader_uuid: "4a928a92552a4e43ba96973316bdf77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a928a92552a4e43ba96973316bdf77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } } }
I20260928 06:44:41.934069 29962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:41.934442 29964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a928a92552a4e43ba96973316bdf77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a928a92552a4e43ba96973316bdf77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } } }
I20260928 06:44:41.934540 29964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:41.937991 29977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:41.938076 29977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:41.938462 29973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:41.939445 29973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:41.945621 29973 catalog_manager.cc:1347] Generated new cluster ID: 216022852ea54e35b5317a59f4646b21
I20260928 06:44:41.945686 29973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:41.968098 29973 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:41.969343 29973 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:41.973986 29973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a928a92552a4e43ba96973316bdf77a: Generated new TSK 0
I20260928 06:44:41.974819 29973 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:41.986653 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 29883
W20260928 06:44:41.992218 29868 connection.cc:535] client connection to 127.0.0.1:43761 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43761): Connection reset by peer (error 104)
2026-09-28T06:44:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:42 dist-test-slave-gtr8 krb5kdc[29983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:42 dist-test-slave-gtr8 krb5kdc[29983](info): set up 2 sockets
Sep 28 06:44:42 dist-test-slave-gtr8 krb5kdc[29983](info): commencing operation
krb5kdc: starting...
W20260928 06:44:42.963616 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:43 dist-test-slave-gtr8 krb5kdc[29983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:43.086616 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38945
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:43.199187 29996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:43.199510 29996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:43.199581 29996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:43.203725 29996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:43.203828 29996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:43.203860 29996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:43.203891 29996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:43.203925 29996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:43.209630 29996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:43.210779 29996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:43.211861 29996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:43.221411 30002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:43.221442 30004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:43.221398 30001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:43.221647 29996 server_base.cc:1034] running on GCE node
I20260928 06:44:43.222206 29996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:43.223471 29996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:43.224658 29996 hybrid_clock.cc:648] HybridClock initialized: now 1790577883224635 us; error 40 us; skew 500 ppm
Sep 28 06:44:43 dist-test-slave-gtr8 krb5kdc[29983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577883, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:43.227509 29996 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:43.228380 29996 webserver.cc:460] Webserver started at http://127.0.0.1:35469/ using document root <none> and password file <none>
I20260928 06:44:43.228928 29996 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:43.228996 29996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:43.229195 29996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:43.231310 29996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "8085fff0a1694d448e74417255b5d468"
format_stamp: "Formatted at 2026-09-28 06:44:43 on dist-test-slave-gtr8"
server_key: "981eab0b756e55fed52429ff7983041f"
server_key_iv: "8938dc76a362673815d49e24d6454d96"
server_key_version: "encryptionkey@0"
I20260928 06:44:43.231849 29996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "8085fff0a1694d448e74417255b5d468"
format_stamp: "Formatted at 2026-09-28 06:44:43 on dist-test-slave-gtr8"
server_key: "981eab0b756e55fed52429ff7983041f"
server_key_iv: "8938dc76a362673815d49e24d6454d96"
server_key_version: "encryptionkey@0"
I20260928 06:44:43.235723 29996 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260928 06:44:43.238308 30011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:43.239761 29996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:43.239926 29996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "8085fff0a1694d448e74417255b5d468"
format_stamp: "Formatted at 2026-09-28 06:44:43 on dist-test-slave-gtr8"
server_key: "981eab0b756e55fed52429ff7983041f"
server_key_iv: "8938dc76a362673815d49e24d6454d96"
server_key_version: "encryptionkey@0"
I20260928 06:44:43.240036 29996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:43.244591 29996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:43.247632 29996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:43.247792 29996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:43.254740 29996 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38945
I20260928 06:44:43.254788 30064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38945 every 8 connection(s)
I20260928 06:44:43.255707 29996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:43.259310 30065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:43.263365 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 29996
I20260928 06:44:43.263500 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:43.263787 8799 external_mini_cluster.cc:1428] Setting key b23481215f447fd4ff0e03d553a92e35
I20260928 06:44:43.266745 30065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468: Bootstrap starting.
I20260928 06:44:43.269555 30065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:43.269938 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:43.265287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:43125 (local address 127.0.0.1:38945)
0928 06:44:43.265838 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:43.265860 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:43.265911 (+ 51us) server_negotiation.cc:407] Connection header received
0928 06:44:43.266778 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:43.266806 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:43.267237 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:43.267651 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:43.268384 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:43.268390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:43.268407 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:43.268420 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:43.268489 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:43.268519 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:43.268666 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:43.268759 (+ 93us) server_negotiation.cc:299] Negotiation successful
0928 06:44:43.268931 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":158,"threads_started":1}
I20260928 06:44:43.270401 30065 log.cc:824] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:43.272763 30065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468: No bootstrap required, opened a new log
I20260928 06:44:43.273085 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:43.277405 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:43.275664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:56937 (local address 127.0.0.1:38945)
0928 06:44:43.275813 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:43.275819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:43.275834 (+ 15us) server_negotiation.cc:407] Connection header received
0928 06:44:43.276039 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:43.276043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:43.276085 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:43.276177 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:43.276963 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:43.276967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:43.276969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:43.276981 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:43.277010 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:43.277023 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:43.277104 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:43.277212 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 06:44:43.277252 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 06:44:43.277285 30065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8085fff0a1694d448e74417255b5d468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38945 } }
I20260928 06:44:43.277580 30065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:43.277650 30065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8085fff0a1694d448e74417255b5d468, State: Initialized, Role: FOLLOWER
I20260928 06:44:43.278121 30065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8085fff0a1694d448e74417255b5d468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38945 } }
I20260928 06:44:43.278256 30065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:43.278348 30065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:43.278489 30065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:43.279469 30065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8085fff0a1694d448e74417255b5d468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38945 } }
I20260928 06:44:43.279786 30065 leader_election.cc:302] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8085fff0a1694d448e74417255b5d468; no voters:
I20260928 06:44:43.280079 30065 leader_election.cc:288] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:43.280182 30075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:43.280428 30075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [term 1 LEADER]: Becoming Leader. State: Replica: 8085fff0a1694d448e74417255b5d468, State: Running, Role: LEADER
I20260928 06:44:43.280818 30075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8085fff0a1694d448e74417255b5d468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38945 } }
I20260928 06:44:43.281369 30065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:43.282860 30077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8085fff0a1694d448e74417255b5d468. Latest consensus state: current_term: 1 leader_uuid: "8085fff0a1694d448e74417255b5d468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8085fff0a1694d448e74417255b5d468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38945 } } }
I20260928 06:44:43.282910 30076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8085fff0a1694d448e74417255b5d468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8085fff0a1694d448e74417255b5d468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38945 } } }
I20260928 06:44:43.282994 30077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:43.283011 30076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468 [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:43.286306 30090 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:43.286387 30090 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:43.286458 30089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:43.287428 30089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:43.293608 30089 catalog_manager.cc:1347] Generated new cluster ID: 177e1cba94f84402a54ebb00dcc7627d
I20260928 06:44:43.293674 30089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:43.304417 30089 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:43.304677 30089 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:43.311786 30089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8085fff0a1694d448e74417255b5d468: Generated new TSK 0
I20260928 06:44:43.312389 30089 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:43.322760 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 29996
2026-09-28T06:44:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:43 dist-test-slave-gtr8 krb5kdc[30096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:43 dist-test-slave-gtr8 krb5kdc[30096](info): set up 2 sockets
Sep 28 06:44:43 dist-test-slave-gtr8 krb5kdc[30096](info): commencing operation
krb5kdc: starting...
W20260928 06:44:44.292060 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:44 dist-test-slave-gtr8 krb5kdc[30096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:44.414754 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:44.521394 30109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:44.521668 30109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:44.521726 30109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:44.525689 30109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:44.525756 30109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:44.525782 30109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:44.525804 30109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:44.525836 30109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:44.531222 30109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39467
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:44.532241 30109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:44.533215 30109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 06:44:44.541544 30109 server_base.cc:1034] running on GCE node
W20260928 06:44:44.541486 30115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:44.541482 30114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:44.541720 30117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:44.542459 30109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:44.543555 30109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:44.544715 30109 hybrid_clock.cc:648] HybridClock initialized: now 1790577884544689 us; error 44 us; skew 500 ppm
Sep 28 06:44:44 dist-test-slave-gtr8 krb5kdc[30096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577884, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:44.547482 30109 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:44.548326 30109 webserver.cc:460] Webserver started at http://127.0.0.1:36633/ using document root <none> and password file <none>
I20260928 06:44:44.548844 30109 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:44.548915 30109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:44.549129 30109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:44.551126 30109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "7de2889c7ecf44f990daf0910b9bb107"
format_stamp: "Formatted at 2026-09-28 06:44:44 on dist-test-slave-gtr8"
server_key: "8176a0d1ded4ec6cb17b15ff25c0ba95"
server_key_iv: "31041a7768723e26e4b9767ad852f6d3"
server_key_version: "encryptionkey@0"
I20260928 06:44:44.551684 30109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "7de2889c7ecf44f990daf0910b9bb107"
format_stamp: "Formatted at 2026-09-28 06:44:44 on dist-test-slave-gtr8"
server_key: "8176a0d1ded4ec6cb17b15ff25c0ba95"
server_key_iv: "31041a7768723e26e4b9767ad852f6d3"
server_key_version: "encryptionkey@0"
I20260928 06:44:44.555332 30109 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260928 06:44:44.557672 30124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:44.558929 30109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:44:44.559037 30109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "7de2889c7ecf44f990daf0910b9bb107"
format_stamp: "Formatted at 2026-09-28 06:44:44 on dist-test-slave-gtr8"
server_key: "8176a0d1ded4ec6cb17b15ff25c0ba95"
server_key_iv: "31041a7768723e26e4b9767ad852f6d3"
server_key_version: "encryptionkey@0"
I20260928 06:44:44.559127 30109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:44.574126 30109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:44.577193 30109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:44.577371 30109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:44.584723 30109 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39467
I20260928 06:44:44.584766 30176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39467 every 8 connection(s)
I20260928 06:44:44.585656 30109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:44.588353 30177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:44.590724 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 30109
I20260928 06:44:44.590873 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:44.591214 8799 external_mini_cluster.cc:1428] Setting key ab5c8afbf4fec6469b513fd50fea90bf
I20260928 06:44:44.594383 30177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107: Bootstrap starting.
I20260928 06:44:44.596958 30177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:44.597734 30177 log.cc:824] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:44.599896 30177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107: No bootstrap required, opened a new log
Sep 28 06:44:44 dist-test-slave-gtr8 krb5kdc[30096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 06:44:44.603659 30177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de2889c7ecf44f990daf0910b9bb107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39467 } }
I20260928 06:44:44.603921 30177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:44.603996 30177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7de2889c7ecf44f990daf0910b9bb107, State: Initialized, Role: FOLLOWER
I20260928 06:44:44.604460 30177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de2889c7ecf44f990daf0910b9bb107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39467 } }
I20260928 06:44:44.604606 30177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:44.604697 30177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:44.604794 30177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:44.605818 30177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de2889c7ecf44f990daf0910b9bb107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39467 } }
I20260928 06:44:44.606161 30177 leader_election.cc:302] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7de2889c7ecf44f990daf0910b9bb107; no voters:
I20260928 06:44:44.606468 30177 leader_election.cc:288] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:44.606599 30182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:44.606604 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:44.592727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:42493 (local address 127.0.0.1:39467)
0928 06:44:44.593208 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:44.593227 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:44.593269 (+ 42us) server_negotiation.cc:407] Connection header received
0928 06:44:44.594063 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:44.594085 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:44.594509 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:44.594849 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:44.595775 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:44.597216 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:44.598323 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:44.598840 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:44.599001 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:44.601434 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:44.601439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:44.601451 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:44.601476 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:44.603555 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:44.604048 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:44.604054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:44.604060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:44.604181 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:44.604522 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:44.604524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:44.604525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:44.604996 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:44.605195 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:44.605496 (+ 301us) server_negotiation.cc:299] Negotiation successful
0928 06:44:44.605723 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":143,"threads_started":1}
I20260928 06:44:44.606874 30182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [term 1 LEADER]: Becoming Leader. State: Replica: 7de2889c7ecf44f990daf0910b9bb107, State: Running, Role: LEADER
I20260928 06:44:44.607228 30182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de2889c7ecf44f990daf0910b9bb107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39467 } }
I20260928 06:44:44.607695 30177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:44.608210 30182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7de2889c7ecf44f990daf0910b9bb107. Latest consensus state: current_term: 1 leader_uuid: "7de2889c7ecf44f990daf0910b9bb107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de2889c7ecf44f990daf0910b9bb107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39467 } } }
I20260928 06:44:44.608311 30182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:44.608647 30184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7de2889c7ecf44f990daf0910b9bb107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de2889c7ecf44f990daf0910b9bb107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39467 } } }
I20260928 06:44:44.608791 30184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:44.610086 30189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:44.611694 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:44.613025 30189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:44.619190 30189 catalog_manager.cc:1347] Generated new cluster ID: b14f3bcbbf094e3db63171157ff37014
I20260928 06:44:44.619243 30189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:44.623934 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:44.613794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:40549 (local address 127.0.0.1:39467)
0928 06:44:44.613902 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:44.613906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:44.614644 (+ 738us) server_negotiation.cc:407] Connection header received
0928 06:44:44.614821 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:44.614824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:44.614872 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:44.614920 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:44.615854 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:44.616668 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:44.617867 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:44.618223 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:44.618296 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:44.620094 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:44.620098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:44.620101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:44.620126 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:44.621809 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:44.622382 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:44.622386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:44.622388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:44.622483 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:44.622865 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:44.622868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:44.622870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:44.623073 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:44.623239 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:44.623681 (+ 442us) server_negotiation.cc:299] Negotiation successful
0928 06:44:44.623777 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 06:44:44.632918 30189 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:44.633980 30189 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:44.648162 30189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7de2889c7ecf44f990daf0910b9bb107: Generated new TSK 0
I20260928 06:44:44.648981 30189 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:44.654183 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 30109
W20260928 06:44:44.661450 30094 connection.cc:535] client connection to 127.0.0.1:39467 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39467): Connection reset by peer (error 104)
2026-09-28T06:44:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:44 dist-test-slave-gtr8 krb5kdc[30209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:44 dist-test-slave-gtr8 krb5kdc[30209](info): set up 2 sockets
Sep 28 06:44:44 dist-test-slave-gtr8 krb5kdc[30209](info): commencing operation
krb5kdc: starting...
W20260928 06:44:45.648106 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:45 dist-test-slave-gtr8 krb5kdc[30209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:45.776418 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:45.899745 30222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:45.900003 30222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:45.900043 30222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:45.904054 30222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:45.904112 30222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:45.904131 30222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:45.904146 30222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:45.904163 30222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:45.909322 30222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36269
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:45.910332 30222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:45.911360 30222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:45.919446 30228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:45.919486 30227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:45.919486 30230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:45.919799 30222 server_base.cc:1034] running on GCE node
I20260928 06:44:45.920377 30222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:45.921643 30222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:45.922827 30222 hybrid_clock.cc:648] HybridClock initialized: now 1790577885922800 us; error 35 us; skew 500 ppm
Sep 28 06:44:45 dist-test-slave-gtr8 krb5kdc[30209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577885, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:45.925534 30222 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:45.926370 30222 webserver.cc:460] Webserver started at http://127.0.0.1:43267/ using document root <none> and password file <none>
I20260928 06:44:45.926891 30222 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:45.926957 30222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:45.927189 30222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:45.929319 30222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "76a89a5fad0541e6bc6f85cdd0887d1e"
format_stamp: "Formatted at 2026-09-28 06:44:45 on dist-test-slave-gtr8"
server_key: "f3764bb1145332cdff7e55b1b4c61719"
server_key_iv: "058f441debe5f24acc3f9c877fca29e7"
server_key_version: "encryptionkey@0"
I20260928 06:44:45.929965 30222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "76a89a5fad0541e6bc6f85cdd0887d1e"
format_stamp: "Formatted at 2026-09-28 06:44:45 on dist-test-slave-gtr8"
server_key: "f3764bb1145332cdff7e55b1b4c61719"
server_key_iv: "058f441debe5f24acc3f9c877fca29e7"
server_key_version: "encryptionkey@0"
I20260928 06:44:45.934423 30222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260928 06:44:45.937180 30237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:45.938580 30222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260928 06:44:45.938694 30222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "76a89a5fad0541e6bc6f85cdd0887d1e"
format_stamp: "Formatted at 2026-09-28 06:44:45 on dist-test-slave-gtr8"
server_key: "f3764bb1145332cdff7e55b1b4c61719"
server_key_iv: "058f441debe5f24acc3f9c877fca29e7"
server_key_version: "encryptionkey@0"
I20260928 06:44:45.938796 30222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:45.960234 30222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:45.963696 30222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:45.963876 30222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:45.971232 30222 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36269
I20260928 06:44:45.971276 30289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36269 every 8 connection(s)
I20260928 06:44:45.972136 30222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:45.973222 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 30222
I20260928 06:44:45.973351 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:45.973644 8799 external_mini_cluster.cc:1428] Setting key d95c619b3e7918e7d5547f9b9eec3d33
I20260928 06:44:45.975482 30290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:45.982194 30290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e: Bootstrap starting.
I20260928 06:44:45.982489 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:45.974900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:37597 (local address 127.0.0.1:36269)
0928 06:44:45.975311 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:45.975326 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:45.975362 (+ 36us) server_negotiation.cc:407] Connection header received
0928 06:44:45.976186 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:45.976213 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:45.976705 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:45.977058 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:45.977472 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:45.978526 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:45.979448 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:45.979908 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:45.980057 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:45.980883 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:45.980888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:45.980899 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:45.981268 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:45.981318 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:45.981340 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:45.981450 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:45.981635 (+ 185us) server_negotiation.cc:299] Negotiation successful
0928 06:44:45.981764 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":120,"threads_started":1}
I20260928 06:44:45.984902 30290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:45.985261 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:45.985752 30290 log.cc:824] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:45.988034 30290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e: No bootstrap required, opened a new log
I20260928 06:44:45.991819 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:45.987332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:45819 (local address 127.0.0.1:36269)
0928 06:44:45.987443 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:45.987446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:45.987633 (+ 187us) server_negotiation.cc:407] Connection header received
0928 06:44:45.987735 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:45.987738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:45.987785 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:45.987861 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:45.988157 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:45.988864 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:45.990012 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:45.990349 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:45.990448 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:45.991335 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:45.991339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:45.991342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:45.991356 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:45.991386 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:45.991399 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:45.991482 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:45.991644 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 06:44:45.991681 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 06:44:45.992403 30290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a89a5fad0541e6bc6f85cdd0887d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } }
I20260928 06:44:45.992626 30290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:45.992708 30290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76a89a5fad0541e6bc6f85cdd0887d1e, State: Initialized, Role: FOLLOWER
I20260928 06:44:45.993168 30290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a89a5fad0541e6bc6f85cdd0887d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } }
I20260928 06:44:45.993295 30290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:45.993346 30290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:45.993443 30290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:45.994257 30290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a89a5fad0541e6bc6f85cdd0887d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } }
I20260928 06:44:45.994558 30290 leader_election.cc:302] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76a89a5fad0541e6bc6f85cdd0887d1e; no voters:
I20260928 06:44:45.994819 30290 leader_election.cc:288] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:45.994932 30300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:45.995203 30300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [term 1 LEADER]: Becoming Leader. State: Replica: 76a89a5fad0541e6bc6f85cdd0887d1e, State: Running, Role: LEADER
I20260928 06:44:45.995573 30300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a89a5fad0541e6bc6f85cdd0887d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } }
I20260928 06:44:45.996007 30290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:45.997424 30301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76a89a5fad0541e6bc6f85cdd0887d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a89a5fad0541e6bc6f85cdd0887d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } } }
I20260928 06:44:45.997589 30301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:45.997890 30302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76a89a5fad0541e6bc6f85cdd0887d1e. Latest consensus state: current_term: 1 leader_uuid: "76a89a5fad0541e6bc6f85cdd0887d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a89a5fad0541e6bc6f85cdd0887d1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } } }
I20260928 06:44:45.997975 30302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:45.998687 30314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:46.001372 30314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:46.006110 30314 catalog_manager.cc:1347] Generated new cluster ID: 36e09e6f092a4df9b20ed9f17508902a
I20260928 06:44:46.006167 30314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:46.021224 30314 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:46.022089 30314 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:46.029629 30314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76a89a5fad0541e6bc6f85cdd0887d1e: Generated new TSK 0
I20260928 06:44:46.030280 30314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:46.037987 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 30222
W20260928 06:44:46.042600 30207 connection.cc:535] client connection to 127.0.0.1:36269 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36269): Connection reset by peer (error 104)
2026-09-28T06:44:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:46 dist-test-slave-gtr8 krb5kdc[30322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:46 dist-test-slave-gtr8 krb5kdc[30322](info): set up 2 sockets
Sep 28 06:44:46 dist-test-slave-gtr8 krb5kdc[30322](info): commencing operation
krb5kdc: starting...
W20260928 06:44:47.007648 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:47 dist-test-slave-gtr8 krb5kdc[30322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 06:44:47.130023 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41249
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:47.236558 30335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:47.236814 30335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:47.236871 30335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:47.240808 30335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:47.240871 30335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:47.240899 30335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:47.240921 30335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:47.240945 30335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:47.246204 30335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41249
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:47.247284 30335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:47.248270 30335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:47.256680 30343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:47.256675 30340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:47.256696 30341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:47.257015 30335 server_base.cc:1034] running on GCE node
I20260928 06:44:47.257727 30335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:47.258827 30335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:47.259991 30335 hybrid_clock.cc:648] HybridClock initialized: now 1790577887259951 us; error 59 us; skew 500 ppm
Sep 28 06:44:47 dist-test-slave-gtr8 krb5kdc[30322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577887, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:47.267522 30335 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 06:44:47.268539 30335 webserver.cc:460] Webserver started at http://127.0.0.1:40655/ using document root <none> and password file <none>
I20260928 06:44:47.269125 30335 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:47.269194 30335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:47.269397 30335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:47.271517 30335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "a22500b41ce34401935e0b796e02aea9"
format_stamp: "Formatted at 2026-09-28 06:44:47 on dist-test-slave-gtr8"
server_key: "49800671fbd33b61c9d49501183cf51b"
server_key_iv: "15e9568849cbe3eac84f05b8a7013e46"
server_key_version: "encryptionkey@0"
I20260928 06:44:47.272059 30335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "a22500b41ce34401935e0b796e02aea9"
format_stamp: "Formatted at 2026-09-28 06:44:47 on dist-test-slave-gtr8"
server_key: "49800671fbd33b61c9d49501183cf51b"
server_key_iv: "15e9568849cbe3eac84f05b8a7013e46"
server_key_version: "encryptionkey@0"
I20260928 06:44:47.275751 30335 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260928 06:44:47.278064 30350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:47.279300 30335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:44:47.279405 30335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "a22500b41ce34401935e0b796e02aea9"
format_stamp: "Formatted at 2026-09-28 06:44:47 on dist-test-slave-gtr8"
server_key: "49800671fbd33b61c9d49501183cf51b"
server_key_iv: "15e9568849cbe3eac84f05b8a7013e46"
server_key_version: "encryptionkey@0"
I20260928 06:44:47.279505 30335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:47.283907 30335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:47.286880 30335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:47.287045 30335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:47.293964 30335 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41249
I20260928 06:44:47.294023 30402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41249 every 8 connection(s)
I20260928 06:44:47.294862 30335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:47.295990 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 30335
I20260928 06:44:47.296132 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:47.296420 8799 external_mini_cluster.cc:1428] Setting key 63aa2c5bd1f9114be3febf2b3216df31
I20260928 06:44:47.297845 30403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:47.301985 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:47.297798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:36985 (local address 127.0.0.1:41249)
0928 06:44:47.298200 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:47.298218 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:47.298259 (+ 41us) server_negotiation.cc:407] Connection header received
0928 06:44:47.299060 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:47.299082 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:47.299456 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:47.299807 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:47.300550 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:47.300556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:47.300571 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:47.300584 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:47.300644 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:47.300671 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:47.300806 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:47.300928 (+ 122us) server_negotiation.cc:299] Negotiation successful
0928 06:44:47.301082 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260928 06:44:47.304557 30403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9: Bootstrap starting.
I20260928 06:44:47.305207 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:47.307228 30403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:47.308073 30403 log.cc:824] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:47.309346 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:47.307801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.61:56813 (local address 127.0.0.1:41249)
0928 06:44:47.307927 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:47.307932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:47.307947 (+ 15us) server_negotiation.cc:407] Connection header received
0928 06:44:47.307984 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:47.307987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:47.308023 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:47.308114 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:47.308916 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:47.308921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:47.308924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:47.308937 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:47.308970 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:47.308983 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:47.309079 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:47.309187 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 06:44:47.309228 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 06:44:47.310392 30403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9: No bootstrap required, opened a new log
I20260928 06:44:47.313874 30403 raft_consensus.cc:348] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22500b41ce34401935e0b796e02aea9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41249 } }
I20260928 06:44:47.314097 30403 raft_consensus.cc:374] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:47.314141 30403 raft_consensus.cc:729] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22500b41ce34401935e0b796e02aea9, State: Initialized, Role: FOLLOWER
I20260928 06:44:47.314543 30403 consensus_queue.cc:260] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22500b41ce34401935e0b796e02aea9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41249 } }
I20260928 06:44:47.314667 30403 raft_consensus.cc:388] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:47.314730 30403 raft_consensus.cc:482] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:47.314822 30403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:47.315887 30403 raft_consensus.cc:504] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22500b41ce34401935e0b796e02aea9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41249 } }
I20260928 06:44:47.316206 30403 leader_election.cc:302] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a22500b41ce34401935e0b796e02aea9; no voters:
I20260928 06:44:47.316489 30403 leader_election.cc:288] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:47.316612 30413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:47.316900 30413 raft_consensus.cc:686] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [term 1 LEADER]: Becoming Leader. State: Replica: a22500b41ce34401935e0b796e02aea9, State: Running, Role: LEADER
I20260928 06:44:47.317281 30413 consensus_queue.cc:237] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22500b41ce34401935e0b796e02aea9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41249 } }
I20260928 06:44:47.317821 30403 sys_catalog.cc:564] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:47.319073 30414 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a22500b41ce34401935e0b796e02aea9. Latest consensus state: current_term: 1 leader_uuid: "a22500b41ce34401935e0b796e02aea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22500b41ce34401935e0b796e02aea9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41249 } } }
I20260928 06:44:47.319264 30414 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:47.319548 30414 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a22500b41ce34401935e0b796e02aea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22500b41ce34401935e0b796e02aea9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41249 } } }
I20260928 06:44:47.319639 30414 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9 [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:47.322988 30428 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:47.323069 30428 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:47.323197 30427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:47.324041 30427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:47.330415 30427 catalog_manager.cc:1347] Generated new cluster ID: d78d1abbbd8344cf8908cd674d13fd47
I20260928 06:44:47.330472 30427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:47.350489 30427 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:47.350835 30427 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:47.357251 30427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a22500b41ce34401935e0b796e02aea9: Generated new TSK 0
I20260928 06:44:47.358031 30427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:47.365520 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 30335
2026-09-28T06:44:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:47 dist-test-slave-gtr8 krb5kdc[30434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:47 dist-test-slave-gtr8 krb5kdc[30434](info): set up 2 sockets
Sep 28 06:44:47 dist-test-slave-gtr8 krb5kdc[30434](info): commencing operation
krb5kdc: starting...
W20260928 06:44:48.327672 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:48 dist-test-slave-gtr8 krb5kdc[30434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:48.448437 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:44557
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:46067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:44557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:48.557824 30447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:48.558096 30447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:48.558154 30447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:48.562165 30447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:48.562222 30447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:48.562248 30447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:48.562271 30447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:48.562304 30447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:48.567610 30447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:46067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:44557
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:44557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:48.568696 30447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:48.569715 30447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:48.577988 30455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:48.578033 30452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:48.578219 30447 server_base.cc:1034] running on GCE node
W20260928 06:44:48.578380 30453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:48.578991 30447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:48.580436 30447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:48.581635 30447 hybrid_clock.cc:648] HybridClock initialized: now 1790577888581606 us; error 37 us; skew 500 ppm
Sep 28 06:44:48 dist-test-slave-gtr8 krb5kdc[30434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577888, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:48.584983 30447 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:48.585824 30447 webserver.cc:460] Webserver started at http://127.8.151.254:46283/ using document root <none> and password file <none>
I20260928 06:44:48.586337 30447 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:48.586395 30447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:48.586576 30447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:48.588594 30447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "78997db293df40399af4f89676a8b23b"
format_stamp: "Formatted at 2026-09-28 06:44:48 on dist-test-slave-gtr8"
server_key: "56edda0031b1652eb5bafe98026ee57d"
server_key_iv: "e3e05a78ba75ca7d0aa6ea45e885e8c7"
server_key_version: "encryptionkey@0"
I20260928 06:44:48.589113 30447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "78997db293df40399af4f89676a8b23b"
format_stamp: "Formatted at 2026-09-28 06:44:48 on dist-test-slave-gtr8"
server_key: "56edda0031b1652eb5bafe98026ee57d"
server_key_iv: "e3e05a78ba75ca7d0aa6ea45e885e8c7"
server_key_version: "encryptionkey@0"
I20260928 06:44:48.593032 30447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260928 06:44:48.595705 30462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:48.597024 30447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:44:48.597143 30447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "78997db293df40399af4f89676a8b23b"
format_stamp: "Formatted at 2026-09-28 06:44:48 on dist-test-slave-gtr8"
server_key: "56edda0031b1652eb5bafe98026ee57d"
server_key_iv: "e3e05a78ba75ca7d0aa6ea45e885e8c7"
server_key_version: "encryptionkey@0"
I20260928 06:44:48.597237 30447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:48.608783 30447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:48.611651 30447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:48.611814 30447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:48.619026 30447 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:44557
I20260928 06:44:48.619076 30515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:44557 every 8 connection(s)
I20260928 06:44:48.620000 30447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:48.623452 30516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:48.624019 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 30447
I20260928 06:44:48.624153 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:48.624435 8799 external_mini_cluster.cc:1428] Setting key 7cc7f02a1b9b4f049f90d4b22844cf57
I20260928 06:44:48.630648 30516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b: Bootstrap starting.
Sep 28 06:44:48 dist-test-slave-gtr8 krb5kdc[30434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:48.633404 30516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:48.634277 30516 log.cc:824] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:48.636477 30516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b: No bootstrap required, opened a new log
I20260928 06:44:48.639032 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:48.626354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39805 (local address 127.8.151.254:44557)
0928 06:44:48.626804 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:48.626819 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:48.626854 (+ 35us) server_negotiation.cc:407] Connection header received
0928 06:44:48.627517 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:48.627534 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:48.627862 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:48.628169 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:48.628924 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:48.629884 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:48.630714 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:48.631262 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:48.631405 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:48.633758 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:48.633763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:48.633776 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:48.633801 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:48.635588 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:48.636175 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:48.636181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:48.636186 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:48.636320 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:48.636653 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:48.636657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:48.636658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:48.637140 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:48.637327 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:48.637670 (+ 343us) server_negotiation.cc:299] Negotiation successful
0928 06:44:48.638072 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":118,"threads_started":1}
I20260928 06:44:48.640709 30516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78997db293df40399af4f89676a8b23b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44557 } }
I20260928 06:44:48.640924 30516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:48.641003 30516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78997db293df40399af4f89676a8b23b, State: Initialized, Role: FOLLOWER
I20260928 06:44:48.641438 30516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78997db293df40399af4f89676a8b23b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44557 } }
I20260928 06:44:48.641600 30516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:48.641655 30516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:48.641727 30516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:48.642423 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:48.642782 30516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78997db293df40399af4f89676a8b23b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44557 } }
I20260928 06:44:48.643186 30516 leader_election.cc:302] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78997db293df40399af4f89676a8b23b; no voters:
I20260928 06:44:48.643532 30516 leader_election.cc:288] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:48.643652 30521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:48.643918 30521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [term 1 LEADER]: Becoming Leader. State: Replica: 78997db293df40399af4f89676a8b23b, State: Running, Role: LEADER
I20260928 06:44:48.644258 30521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78997db293df40399af4f89676a8b23b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44557 } }
I20260928 06:44:48.644862 30516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:48.645622 30527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78997db293df40399af4f89676a8b23b. Latest consensus state: current_term: 1 leader_uuid: "78997db293df40399af4f89676a8b23b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78997db293df40399af4f89676a8b23b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44557 } } }
I20260928 06:44:48.645764 30527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:48.645987 30525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78997db293df40399af4f89676a8b23b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78997db293df40399af4f89676a8b23b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44557 } } }
I20260928 06:44:48.646155 30525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:48.650031 30541 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:48.650106 30541 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:48.650163 30535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:48.650985 30535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:48.652673 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:48.644580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55781 (local address 127.8.151.254:44557)
0928 06:44:48.644677 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:48.644681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:48.645200 (+ 519us) server_negotiation.cc:407] Connection header received
0928 06:44:48.645322 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:48.645325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:48.645369 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:48.645455 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:48.646207 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:48.646947 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:48.648012 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:48.648325 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:48.648418 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:48.649704 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:48.649707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:48.649710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:48.649733 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:48.651041 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:48.651520 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:48.651524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:48.651526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:48.651598 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:48.651899 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:48.651902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:48.651904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:48.652068 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:48.652192 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:48.652448 (+ 256us) server_negotiation.cc:299] Negotiation successful
0928 06:44:48.652535 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 06:44:48.656497 30535 catalog_manager.cc:1347] Generated new cluster ID: 6e7a3a7bb0fc42809bee02c10075c6e3
I20260928 06:44:48.656553 30535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:48.667478 30535 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:48.668323 30535 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:48.672461 30535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78997db293df40399af4f89676a8b23b: Generated new TSK 0
I20260928 06:44:48.673051 30535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:48.678361 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 30447
W20260928 06:44:48.683039 30432 connection.cc:535] client connection to 127.8.151.254:44557 recv error: Network error: failed to read from TLS socket (remote: 127.8.151.254:44557): Connection reset by peer (error 104)
2026-09-28T06:44:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:48 dist-test-slave-gtr8 krb5kdc[30547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:48 dist-test-slave-gtr8 krb5kdc[30547](info): set up 2 sockets
Sep 28 06:44:48 dist-test-slave-gtr8 krb5kdc[30547](info): commencing operation
krb5kdc: starting...
W20260928 06:44:49.707705 8799 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:49 dist-test-slave-gtr8 krb5kdc[30547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:49.830237 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:40055
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:37701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:40055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:49.938763 30560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:49.939013 30560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:49.939055 30560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:49.943301 30560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:49.943382 30560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:49.943411 30560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:49.943432 30560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:49.943454 30560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:49.948936 30560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:37701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:40055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:40055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:49.949934 30560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:49.950934 30560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:49.959251 30565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:49.959342 30560 server_base.cc:1034] running on GCE node
W20260928 06:44:49.959198 30568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:49.959465 30566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:49.959986 30560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:49.961081 30560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:49.962261 30560 hybrid_clock.cc:648] HybridClock initialized: now 1790577889962247 us; error 34 us; skew 500 ppm
Sep 28 06:44:49 dist-test-slave-gtr8 krb5kdc[30547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577889, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:49.965102 30560 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:49.966077 30560 webserver.cc:460] Webserver started at http://127.8.151.254:45623/ using document root <none> and password file <none>
I20260928 06:44:49.966627 30560 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:49.966689 30560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:49.966898 30560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:49.969017 30560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "9744efd4ae194ead9e29214802484e4b"
format_stamp: "Formatted at 2026-09-28 06:44:49 on dist-test-slave-gtr8"
server_key: "492e7d9dd5c12b9178ef2c4bb894afdb"
server_key_iv: "201c200644ba734eb94f1cfaa8f18f23"
server_key_version: "encryptionkey@0"
I20260928 06:44:49.969563 30560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "9744efd4ae194ead9e29214802484e4b"
format_stamp: "Formatted at 2026-09-28 06:44:49 on dist-test-slave-gtr8"
server_key: "492e7d9dd5c12b9178ef2c4bb894afdb"
server_key_iv: "201c200644ba734eb94f1cfaa8f18f23"
server_key_version: "encryptionkey@0"
I20260928 06:44:49.973523 30560 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:44:49.976039 30575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:49.977356 30560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:44:49.977485 30560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "9744efd4ae194ead9e29214802484e4b"
format_stamp: "Formatted at 2026-09-28 06:44:49 on dist-test-slave-gtr8"
server_key: "492e7d9dd5c12b9178ef2c4bb894afdb"
server_key_iv: "201c200644ba734eb94f1cfaa8f18f23"
server_key_version: "encryptionkey@0"
I20260928 06:44:49.977583 30560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:50.000710 30560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:50.003837 30560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:50.004017 30560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:50.010732 30560 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:40055
I20260928 06:44:50.010772 30628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:40055 every 8 connection(s)
I20260928 06:44:50.011766 30560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:50.015067 30629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:50.016734 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 30560
I20260928 06:44:50.016876 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:50.017170 8799 external_mini_cluster.cc:1428] Setting key 630457b7ffeb01bb52c5066192be85f1
I20260928 06:44:50.021950 30629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b: Bootstrap starting.
I20260928 06:44:50.024529 30629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:50.025375 30632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:50.018426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37819 (local address 127.8.151.254:40055)
0928 06:44:50.018807 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:50.018822 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:50.018856 (+ 34us) server_negotiation.cc:407] Connection header received
0928 06:44:50.019501 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:50.019519 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:50.019850 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:50.020152 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:50.020525 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:50.021466 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:50.022350 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:50.022806 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:50.022957 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:50.023788 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:50.023793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:50.023805 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:50.024175 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:50.024224 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:50.024246 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:50.024358 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:50.024540 (+ 182us) server_negotiation.cc:299] Negotiation successful
0928 06:44:50.024676 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":107,"threads_started":1}
I20260928 06:44:50.025432 30629 log.cc:824] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:50.027642 30629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b: No bootstrap required, opened a new log
I20260928 06:44:50.028062 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:50.032310 30629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9744efd4ae194ead9e29214802484e4b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40055 } }
I20260928 06:44:50.032608 30629 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:50.032678 30629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9744efd4ae194ead9e29214802484e4b, State: Initialized, Role: FOLLOWER
I20260928 06:44:50.033267 30629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9744efd4ae194ead9e29214802484e4b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40055 } }
I20260928 06:44:50.033421 30632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:50.029878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58683 (local address 127.8.151.254:40055)
0928 06:44:50.029998 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:50.030003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:50.030140 (+ 137us) server_negotiation.cc:407] Connection header received
0928 06:44:50.030256 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:50.030258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:50.030291 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:50.030357 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:50.030616 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:50.031106 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:50.031898 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:50.032142 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:50.032225 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:50.032996 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:50.033000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:50.033003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:50.033016 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:50.033042 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:50.033055 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:50.033127 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:50.033264 (+ 137us) server_negotiation.cc:299] Negotiation successful
0928 06:44:50.033293 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 06:44:50.033492 30629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:50.033602 30629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:50.033716 30629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:50.034895 30629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9744efd4ae194ead9e29214802484e4b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40055 } }
I20260928 06:44:50.035313 30629 leader_election.cc:302] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9744efd4ae194ead9e29214802484e4b; no voters:
I20260928 06:44:50.035694 30629 leader_election.cc:288] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:50.035758 30639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:50.035964 30639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [term 1 LEADER]: Becoming Leader. State: Replica: 9744efd4ae194ead9e29214802484e4b, State: Running, Role: LEADER
I20260928 06:44:50.036302 30639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9744efd4ae194ead9e29214802484e4b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40055 } }
I20260928 06:44:50.037096 30629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:50.037858 30640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9744efd4ae194ead9e29214802484e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9744efd4ae194ead9e29214802484e4b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40055 } } }
I20260928 06:44:50.037878 30641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9744efd4ae194ead9e29214802484e4b. Latest consensus state: current_term: 1 leader_uuid: "9744efd4ae194ead9e29214802484e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9744efd4ae194ead9e29214802484e4b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40055 } } }
I20260928 06:44:50.038017 30640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:50.038077 30641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:50.041999 30654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:50.042063 30654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:50.042130 30655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:50.043337 30655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:50.049451 30655 catalog_manager.cc:1347] Generated new cluster ID: 098ddc4edb744e63b380791863ede353
I20260928 06:44:50.049517 30655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:50.067955 30655 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:50.069278 30655 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:50.080924 30655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9744efd4ae194ead9e29214802484e4b: Generated new TSK 0
I20260928 06:44:50.081781 30655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:50.092707 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 30560
W20260928 06:44:50.097405 30545 connection.cc:535] client connection to 127.8.151.254:40055 recv error: Network error: failed to read from TLS socket (remote: 127.8.151.254:40055): Connection reset by peer (error 104)
2026-09-28T06:44:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:50 dist-test-slave-gtr8 krb5kdc[30660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:50 dist-test-slave-gtr8 krb5kdc[30660](info): set up 2 sockets
Sep 28 06:44:50 dist-test-slave-gtr8 krb5kdc[30660](info): commencing operation
krb5kdc: starting...
W20260928 06:44:51.063920 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:51 dist-test-slave-gtr8 krb5kdc[30660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:51.197911 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:44803
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:35365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:44803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:51.308686 30673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:51.308952 30673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:51.309011 30673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:51.313788 30673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:51.313875 30673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:51.313905 30673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:51.313926 30673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:51.313957 30673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:51.320430 30673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:35365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:44803
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:44803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:51.321559 30673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:51.322602 30673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:51.330850 30681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:51.330941 30673 server_base.cc:1034] running on GCE node
W20260928 06:44:51.331130 30678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:51.330850 30679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:51.331800 30673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:51.332923 30673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:51.334081 30673 hybrid_clock.cc:648] HybridClock initialized: now 1790577891334044 us; error 53 us; skew 500 ppm
Sep 28 06:44:51 dist-test-slave-gtr8 krb5kdc[30660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577891, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:51.336730 30673 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:51.337584 30673 webserver.cc:460] Webserver started at http://127.8.151.254:39647/ using document root <none> and password file <none>
I20260928 06:44:51.338117 30673 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:51.338181 30673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:51.338383 30673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:51.340543 30673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "38d736286dc94409a1fda41505e0523b"
format_stamp: "Formatted at 2026-09-28 06:44:51 on dist-test-slave-gtr8"
server_key: "1e2f164ad3b6a6d55e055430cbc81f10"
server_key_iv: "04f3c6ec62252c5e803703b68ec286bf"
server_key_version: "encryptionkey@0"
I20260928 06:44:51.341095 30673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "38d736286dc94409a1fda41505e0523b"
format_stamp: "Formatted at 2026-09-28 06:44:51 on dist-test-slave-gtr8"
server_key: "1e2f164ad3b6a6d55e055430cbc81f10"
server_key_iv: "04f3c6ec62252c5e803703b68ec286bf"
server_key_version: "encryptionkey@0"
I20260928 06:44:51.344893 30673 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:44:51.347353 30688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:51.348662 30673 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:44:51.348776 30673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "38d736286dc94409a1fda41505e0523b"
format_stamp: "Formatted at 2026-09-28 06:44:51 on dist-test-slave-gtr8"
server_key: "1e2f164ad3b6a6d55e055430cbc81f10"
server_key_iv: "04f3c6ec62252c5e803703b68ec286bf"
server_key_version: "encryptionkey@0"
I20260928 06:44:51.348872 30673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:51.353333 30673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:51.356505 30673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:51.356669 30673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:51.363864 30673 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:44803
I20260928 06:44:51.363893 30741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:44803 every 8 connection(s)
I20260928 06:44:51.364780 30673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:51.367475 30742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:51.372841 30742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b: Bootstrap starting.
I20260928 06:44:51.374053 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 30673
I20260928 06:44:51.374193 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:51.374473 8799 external_mini_cluster.cc:1428] Setting key 34053c60f99c8cff742f7e1ae1e2353a
I20260928 06:44:51.375526 30742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:51.376493 30742 log.cc:824] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:51.378787 30742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b: No bootstrap required, opened a new log
I20260928 06:44:51.380024 30745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:51.375655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53117 (local address 127.8.151.254:44803)
0928 06:44:51.376111 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:51.376129 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:51.376169 (+ 40us) server_negotiation.cc:407] Connection header received
0928 06:44:51.377009 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:51.377029 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:51.377326 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:51.377682 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:51.378415 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:51.378420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:51.378434 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:51.378446 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:51.378512 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:51.378540 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:51.378650 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:51.378780 (+ 130us) server_negotiation.cc:299] Negotiation successful
0928 06:44:51.379083 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":136,"threads_started":1}
I20260928 06:44:51.383025 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:51.383095 30742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d736286dc94409a1fda41505e0523b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44803 } }
I20260928 06:44:51.383361 30742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:51.383453 30742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d736286dc94409a1fda41505e0523b, State: Initialized, Role: FOLLOWER
I20260928 06:44:51.383893 30742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d736286dc94409a1fda41505e0523b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44803 } }
I20260928 06:44:51.384028 30742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:51.384091 30742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:51.384187 30742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:51.385219 30742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d736286dc94409a1fda41505e0523b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44803 } }
I20260928 06:44:51.385594 30742 leader_election.cc:302] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38d736286dc94409a1fda41505e0523b; no voters:
I20260928 06:44:51.385900 30742 leader_election.cc:288] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:51.385969 30752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:51.386202 30752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [term 1 LEADER]: Becoming Leader. State: Replica: 38d736286dc94409a1fda41505e0523b, State: Running, Role: LEADER
I20260928 06:44:51.386538 30752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d736286dc94409a1fda41505e0523b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44803 } }
I20260928 06:44:51.386866 30745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:51.385197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40687 (local address 127.8.151.254:44803)
0928 06:44:51.385297 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:51.385301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:51.385524 (+ 223us) server_negotiation.cc:407] Connection header received
0928 06:44:51.385627 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:51.385630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:51.385662 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:51.385724 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:51.386454 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:51.386457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:51.386459 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:51.386470 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:51.386496 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:51.386519 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:51.386571 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:51.386723 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 06:44:51.386762 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 06:44:51.387281 30742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:51.388605 30752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38d736286dc94409a1fda41505e0523b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d736286dc94409a1fda41505e0523b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44803 } } }
I20260928 06:44:51.388780 30752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:51.389019 30752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38d736286dc94409a1fda41505e0523b. Latest consensus state: current_term: 1 leader_uuid: "38d736286dc94409a1fda41505e0523b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d736286dc94409a1fda41505e0523b" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 44803 } } }
I20260928 06:44:51.389166 30752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:51.392257 30767 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:51.392323 30767 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:51.392410 30763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:51.393142 30763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:51.398224 30763 catalog_manager.cc:1347] Generated new cluster ID: f1a3bfd44dae48c9b999214a65037c9b
I20260928 06:44:51.398288 30763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:51.406482 30763 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:51.406823 30763 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:51.417382 30763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38d736286dc94409a1fda41505e0523b: Generated new TSK 0
I20260928 06:44:51.418074 30763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:51.426776 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 30673
2026-09-28T06:44:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:51 dist-test-slave-gtr8 krb5kdc[30773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:51 dist-test-slave-gtr8 krb5kdc[30773](info): set up 2 sockets
Sep 28 06:44:51 dist-test-slave-gtr8 krb5kdc[30773](info): commencing operation
krb5kdc: starting...
W20260928 06:44:52.427574 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:52 dist-test-slave-gtr8 krb5kdc[30773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:52.550043 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:37347
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:42257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:37347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:52.661604 30786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:52.661866 30786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:52.661907 30786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:52.665989 30786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:52.666047 30786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:52.666067 30786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:52.666082 30786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:52.666100 30786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:52.671383 30786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:42257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:37347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:37347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:52.672461 30786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:52.673465 30786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:52.681643 30791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:52.681669 30794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:52.681790 30786 server_base.cc:1034] running on GCE node
W20260928 06:44:52.681669 30792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:52.682511 30786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:52.683676 30786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:52.684851 30786 hybrid_clock.cc:648] HybridClock initialized: now 1790577892684826 us; error 44 us; skew 500 ppm
Sep 28 06:44:52 dist-test-slave-gtr8 krb5kdc[30773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577892, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:52.693145 30786 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:52.694108 30786 webserver.cc:460] Webserver started at http://127.8.151.254:43517/ using document root <none> and password file <none>
I20260928 06:44:52.694739 30786 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:52.694825 30786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:52.695058 30786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:52.697789 30786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "59518470b2a64d0da71f9d6721b01cb5"
format_stamp: "Formatted at 2026-09-28 06:44:52 on dist-test-slave-gtr8"
server_key: "3ff25b1555cd0273f02883e693f3ba57"
server_key_iv: "e811894369db1892dc22d7289a939bb2"
server_key_version: "encryptionkey@0"
I20260928 06:44:52.698436 30786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "59518470b2a64d0da71f9d6721b01cb5"
format_stamp: "Formatted at 2026-09-28 06:44:52 on dist-test-slave-gtr8"
server_key: "3ff25b1555cd0273f02883e693f3ba57"
server_key_iv: "e811894369db1892dc22d7289a939bb2"
server_key_version: "encryptionkey@0"
I20260928 06:44:52.703016 30786 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260928 06:44:52.705967 30801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:52.707436 30786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:52.707562 30786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "59518470b2a64d0da71f9d6721b01cb5"
format_stamp: "Formatted at 2026-09-28 06:44:52 on dist-test-slave-gtr8"
server_key: "3ff25b1555cd0273f02883e693f3ba57"
server_key_iv: "e811894369db1892dc22d7289a939bb2"
server_key_version: "encryptionkey@0"
I20260928 06:44:52.707669 30786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:52.721532 30786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:52.724659 30786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:52.724828 30786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:52.733067 30853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:37347 every 8 connection(s)
I20260928 06:44:52.733078 30786 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:37347
I20260928 06:44:52.734355 30786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:52.736594 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 30786
I20260928 06:44:52.736742 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:52.737022 8799 external_mini_cluster.cc:1428] Setting key 15d8713f7fe72859da02a9ccb9d9907d
I20260928 06:44:52.737317 30854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:52.744197 30854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5: Bootstrap starting.
I20260928 06:44:52.746721 30854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:52.747540 30854 log.cc:824] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5: Log is configured to *not* fsync() on all Append() calls
Sep 28 06:44:52 dist-test-slave-gtr8 krb5kdc[30773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:52.749728 30854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5: No bootstrap required, opened a new log
I20260928 06:44:52.752497 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:52.738469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47793 (local address 127.8.151.254:37347)
0928 06:44:52.738886 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:52.738905 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:52.738946 (+ 41us) server_negotiation.cc:407] Connection header received
0928 06:44:52.739932 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:52.739956 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:52.740397 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:52.740785 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:52.741632 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:52.743107 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:52.744378 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:52.745090 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:52.745259 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:52.748226 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:52.748233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:52.748249 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:52.748279 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:52.749782 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:52.750211 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:52.750217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:52.750223 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:52.750330 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:52.750577 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:52.750580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:52.750581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:52.751009 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:52.751225 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:52.751521 (+ 296us) server_negotiation.cc:299] Negotiation successful
0928 06:44:52.751806 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":118,"threads_started":1}
I20260928 06:44:52.754237 30854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59518470b2a64d0da71f9d6721b01cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 37347 } }
I20260928 06:44:52.754523 30854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:52.754628 30854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59518470b2a64d0da71f9d6721b01cb5, State: Initialized, Role: FOLLOWER
I20260928 06:44:52.755082 30854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59518470b2a64d0da71f9d6721b01cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 37347 } }
I20260928 06:44:52.755249 30854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:52.755319 30854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:52.755412 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:52.755451 30854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:52.756539 30854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59518470b2a64d0da71f9d6721b01cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 37347 } }
I20260928 06:44:52.756908 30854 leader_election.cc:302] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59518470b2a64d0da71f9d6721b01cb5; no voters:
I20260928 06:44:52.757228 30854 leader_election.cc:288] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:52.757803 30865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:52.758028 30865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [term 1 LEADER]: Becoming Leader. State: Replica: 59518470b2a64d0da71f9d6721b01cb5, State: Running, Role: LEADER
I20260928 06:44:52.758339 30865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59518470b2a64d0da71f9d6721b01cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 37347 } }
I20260928 06:44:52.758535 30854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:52.759918 30867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59518470b2a64d0da71f9d6721b01cb5. Latest consensus state: current_term: 1 leader_uuid: "59518470b2a64d0da71f9d6721b01cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59518470b2a64d0da71f9d6721b01cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 37347 } } }
I20260928 06:44:52.760038 30867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:52.760296 30864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59518470b2a64d0da71f9d6721b01cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59518470b2a64d0da71f9d6721b01cb5" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 37347 } } }
I20260928 06:44:52.760370 30864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5 [sys.catalog]: This master's current role is: LEADER
W20260928 06:44:52.763273 30880 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:52.763397 30880 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:52.763497 30878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:52.764324 30878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:52.769374 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:52.757597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55717 (local address 127.8.151.254:37347)
0928 06:44:52.757694 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:52.757698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:52.760907 (+ 3209us) server_negotiation.cc:407] Connection header received
0928 06:44:52.761088 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:52.761092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:52.761136 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:52.761183 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:52.762056 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:52.762807 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:52.763929 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:52.764257 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:52.764331 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:52.766008 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:52.766011 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:52.766014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:52.766038 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:52.767565 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:52.768042 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:52.768045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:52.768047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:52.768130 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:52.768417 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:52.768421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:52.768423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:52.768643 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:52.768744 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:52.769100 (+ 356us) server_negotiation.cc:299] Negotiation successful
0928 06:44:52.769218 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 06:44:52.776461 30878 catalog_manager.cc:1347] Generated new cluster ID: 5236ef10658f4280841f0f2997050b22
I20260928 06:44:52.776536 30878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:52.790499 30878 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:52.791728 30878 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:52.796765 30878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59518470b2a64d0da71f9d6721b01cb5: Generated new TSK 0
I20260928 06:44:52.797436 30878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:52.805154 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 30786
2026-09-28T06:44:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:52 dist-test-slave-gtr8 krb5kdc[30886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:52 dist-test-slave-gtr8 krb5kdc[30886](info): set up 2 sockets
Sep 28 06:44:52 dist-test-slave-gtr8 krb5kdc[30886](info): commencing operation
krb5kdc: starting...
W20260928 06:44:53.808964 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:53 dist-test-slave-gtr8 krb5kdc[30886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:53.933895 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:40561
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:40561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:54.042793 30899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:54.043058 30899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:54.043116 30899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:54.046988 30899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:54.047050 30899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:54.047076 30899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:54.047099 30899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:54.047127 30899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:54.052335 30899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:40561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:40561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:54.053373 30899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:54.054384 30899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:54.062729 30905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:54.062767 30899 server_base.cc:1034] running on GCE node
W20260928 06:44:54.062819 30904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:54.062798 30907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:54.063520 30899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:54.064626 30899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:54.065781 30899 hybrid_clock.cc:648] HybridClock initialized: now 1790577894065758 us; error 41 us; skew 500 ppm
Sep 28 06:44:54 dist-test-slave-gtr8 krb5kdc[30886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577894, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:54.068543 30899 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:54.069484 30899 webserver.cc:460] Webserver started at http://127.8.151.254:46849/ using document root <none> and password file <none>
I20260928 06:44:54.070004 30899 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:54.070071 30899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:54.070261 30899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:54.072320 30899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "41b3bbcb00e8494dbdd7d670d4aea58e"
format_stamp: "Formatted at 2026-09-28 06:44:54 on dist-test-slave-gtr8"
server_key: "f5656b793c245eea525e91f30a6a4fe7"
server_key_iv: "fcb6ea7aacb7475cd1d9427a8a460e8b"
server_key_version: "encryptionkey@0"
I20260928 06:44:54.072853 30899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "41b3bbcb00e8494dbdd7d670d4aea58e"
format_stamp: "Formatted at 2026-09-28 06:44:54 on dist-test-slave-gtr8"
server_key: "f5656b793c245eea525e91f30a6a4fe7"
server_key_iv: "fcb6ea7aacb7475cd1d9427a8a460e8b"
server_key_version: "encryptionkey@0"
I20260928 06:44:54.076587 30899 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 06:44:54.079007 30914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:54.080410 30899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:44:54.080536 30899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "41b3bbcb00e8494dbdd7d670d4aea58e"
format_stamp: "Formatted at 2026-09-28 06:44:54 on dist-test-slave-gtr8"
server_key: "f5656b793c245eea525e91f30a6a4fe7"
server_key_iv: "fcb6ea7aacb7475cd1d9427a8a460e8b"
server_key_version: "encryptionkey@0"
I20260928 06:44:54.080646 30899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:54.092852 30899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:54.095923 30899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:54.096109 30899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:54.102793 30899 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:40561
I20260928 06:44:54.102844 30966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:40561 every 8 connection(s)
I20260928 06:44:54.103765 30899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:54.106366 30967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:54.109982 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 30899
I20260928 06:44:54.110123 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:54.110423 8799 external_mini_cluster.cc:1428] Setting key df4f4153160e74c07874bbd9204065cd
I20260928 06:44:54.112147 30967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e: Bootstrap starting.
I20260928 06:44:54.114768 30967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:54.115655 30967 log.cc:824] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:54.117641 30967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e: No bootstrap required, opened a new log
I20260928 06:44:54.120592 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:54.111815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55773 (local address 127.8.151.254:40561)
0928 06:44:54.112210 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:54.112230 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:54.112271 (+ 41us) server_negotiation.cc:407] Connection header received
0928 06:44:54.113063 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:54.113085 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:54.113528 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:54.113872 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:54.114291 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:54.115764 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:54.116897 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:54.117571 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:54.117737 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:54.118566 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:54.118573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:54.118588 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:54.119080 (+ 492us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:54.119170 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:54.119202 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:54.119293 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:54.119430 (+ 137us) server_negotiation.cc:299] Negotiation successful
0928 06:44:54.119635 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":98,"threads_started":1}
I20260928 06:44:54.121862 30967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b3bbcb00e8494dbdd7d670d4aea58e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40561 } }
I20260928 06:44:54.122083 30967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:54.122162 30967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41b3bbcb00e8494dbdd7d670d4aea58e, State: Initialized, Role: FOLLOWER
I20260928 06:44:54.122685 30967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b3bbcb00e8494dbdd7d670d4aea58e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40561 } }
I20260928 06:44:54.122843 30967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:54.122892 30967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:54.122988 30967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:54.123634 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:54.124096 30967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b3bbcb00e8494dbdd7d670d4aea58e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40561 } }
I20260928 06:44:54.124477 30967 leader_election.cc:302] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41b3bbcb00e8494dbdd7d670d4aea58e; no voters:
I20260928 06:44:54.124848 30967 leader_election.cc:288] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:54.126048 30967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:54.124506 30972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 06:44:54.127230 30972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b3bbcb00e8494dbdd7d670d4aea58e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40561 } }
I20260928 06:44:54.127494 30973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:54.127687 30973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [term 1 LEADER]: Becoming Leader. State: Replica: 41b3bbcb00e8494dbdd7d670d4aea58e, State: Running, Role: LEADER
I20260928 06:44:54.128011 30973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b3bbcb00e8494dbdd7d670d4aea58e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40561 } }
I20260928 06:44:54.128084 30972 leader_election.cc:302] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41b3bbcb00e8494dbdd7d670d4aea58e; no voters:
I20260928 06:44:54.128072 30978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b3bbcb00e8494dbdd7d670d4aea58e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40561 } } }
I20260928 06:44:54.128204 30978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:44:54.128276 30972 leader_election.cc:288] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers
I20260928 06:44:54.129271 30989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41b3bbcb00e8494dbdd7d670d4aea58e. Latest consensus state: current_term: 1 leader_uuid: "41b3bbcb00e8494dbdd7d670d4aea58e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b3bbcb00e8494dbdd7d670d4aea58e" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 40561 } } }
I20260928 06:44:54.129357 30989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:54.129716 30993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:54.132920 30993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:54.133283 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:54.128245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58221 (local address 127.8.151.254:40561)
0928 06:44:54.128347 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:54.128351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:54.128822 (+ 471us) server_negotiation.cc:407] Connection header received
0928 06:44:54.128924 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:54.128927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:54.128972 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:54.129045 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:54.129449 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:54.130198 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:54.131447 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:54.131754 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:54.131827 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:54.132705 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:54.132709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:54.132711 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:54.132724 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:54.132751 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:54.132763 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:54.132822 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:54.133108 (+ 286us) server_negotiation.cc:299] Negotiation successful
0928 06:44:54.133161 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 06:44:54.139389 30993 catalog_manager.cc:1347] Generated new cluster ID: 4e042e42f4f842a09500797cefa642dd
I20260928 06:44:54.139722 30993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:54.164909 30993 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:54.166119 30993 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:54.175103 30993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41b3bbcb00e8494dbdd7d670d4aea58e: Generated new TSK 0
I20260928 06:44:54.175940 30993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:54.184803 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 30899
2026-09-28T06:44:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:54 dist-test-slave-gtr8 krb5kdc[31000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:54 dist-test-slave-gtr8 krb5kdc[31000](info): set up 2 sockets
Sep 28 06:44:54 dist-test-slave-gtr8 krb5kdc[31000](info): commencing operation
krb5kdc: starting...
W20260928 06:44:55.155766 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:55 dist-test-slave-gtr8 krb5kdc[31000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:55.284314 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:34885
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:39139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:34885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:55.417783 31013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:55.418123 31013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:55.418181 31013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:55.423727 31013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:55.423799 31013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:55.423825 31013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:55.423852 31013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:55.423872 31013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:55.431038 31013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:39139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:34885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:34885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:55.432307 31013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:55.433595 31013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:55.442946 31018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:55.442940 31019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:55.443198 31013 server_base.cc:1034] running on GCE node
W20260928 06:44:55.443499 31021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:55.444048 31013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:55.445228 31013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:55.446440 31013 hybrid_clock.cc:648] HybridClock initialized: now 1790577895446420 us; error 41 us; skew 500 ppm
Sep 28 06:44:55 dist-test-slave-gtr8 krb5kdc[31000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577895, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:55.449113 31013 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:55.449992 31013 webserver.cc:460] Webserver started at http://127.8.151.254:45107/ using document root <none> and password file <none>
I20260928 06:44:55.450522 31013 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:55.450579 31013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:55.450762 31013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:55.452801 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "5bb6659840c84170b8ff8df6dd3daab2"
format_stamp: "Formatted at 2026-09-28 06:44:55 on dist-test-slave-gtr8"
server_key: "5a84b008ec2ccf51bbb5710298241760"
server_key_iv: "844dbade07bf22c3a458f98d128983bc"
server_key_version: "encryptionkey@0"
I20260928 06:44:55.453331 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "5bb6659840c84170b8ff8df6dd3daab2"
format_stamp: "Formatted at 2026-09-28 06:44:55 on dist-test-slave-gtr8"
server_key: "5a84b008ec2ccf51bbb5710298241760"
server_key_iv: "844dbade07bf22c3a458f98d128983bc"
server_key_version: "encryptionkey@0"
I20260928 06:44:55.457155 31013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260928 06:44:55.459661 31028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:55.460968 31013 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260928 06:44:55.461086 31013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "5bb6659840c84170b8ff8df6dd3daab2"
format_stamp: "Formatted at 2026-09-28 06:44:55 on dist-test-slave-gtr8"
server_key: "5a84b008ec2ccf51bbb5710298241760"
server_key_iv: "844dbade07bf22c3a458f98d128983bc"
server_key_version: "encryptionkey@0"
I20260928 06:44:55.461200 31013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:55.465692 31013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:55.468698 31013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:55.468864 31013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:55.475238 31013 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:34885
I20260928 06:44:55.475422 31080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:34885 every 8 connection(s)
I20260928 06:44:55.476161 31013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:55.479369 31081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:55.479908 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 31013
I20260928 06:44:55.480022 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:55.480305 8799 external_mini_cluster.cc:1428] Setting key 70ae9a22c606e57b919f5b28b20e3d4a
I20260928 06:44:55.485793 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:55.481613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39015 (local address 127.8.151.254:34885)
0928 06:44:55.482088 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:55.482103 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:55.482140 (+ 37us) server_negotiation.cc:407] Connection header received
0928 06:44:55.483181 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:55.483205 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:55.483480 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:55.483784 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:55.484543 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:55.484548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:55.484560 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:55.484573 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:55.484631 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:55.484650 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:55.484774 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:55.484868 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 06:44:55.485120 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":145,"threads_started":1}
I20260928 06:44:55.487627 31081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2: Bootstrap starting.
I20260928 06:44:55.488266 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:55.490110 31081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:55.490976 31081 log.cc:824] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:55.491631 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:55.490117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37755 (local address 127.8.151.254:34885)
0928 06:44:55.490231 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:55.490235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:55.490442 (+ 207us) server_negotiation.cc:407] Connection header received
0928 06:44:55.490538 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:55.490541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:55.490571 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:55.490643 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:55.491268 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:55.491271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:55.491274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 06:44:55.491284 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:55.491311 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 06:44:55.491323 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:55.491391 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:55.491462 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 06:44:55.491498 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 06:44:55.493126 31081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2: No bootstrap required, opened a new log
I20260928 06:44:55.497295 31081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb6659840c84170b8ff8df6dd3daab2" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34885 } }
I20260928 06:44:55.497570 31081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 06:44:55.497666 31081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bb6659840c84170b8ff8df6dd3daab2, State: Initialized, Role: FOLLOWER
I20260928 06:44:55.498127 31081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb6659840c84170b8ff8df6dd3daab2" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34885 } }
I20260928 06:44:55.498306 31081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 06:44:55.498384 31081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 06:44:55.498482 31081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:55.499599 31081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb6659840c84170b8ff8df6dd3daab2" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34885 } }
I20260928 06:44:55.499949 31081 leader_election.cc:302] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bb6659840c84170b8ff8df6dd3daab2; no voters:
I20260928 06:44:55.500303 31081 leader_election.cc:288] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 06:44:55.500417 31091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:55.500674 31091 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [term 1 LEADER]: Becoming Leader. State: Replica: 5bb6659840c84170b8ff8df6dd3daab2, State: Running, Role: LEADER
I20260928 06:44:55.501003 31091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb6659840c84170b8ff8df6dd3daab2" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34885 } }
I20260928 06:44:55.501822 31081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:55.502264 31091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bb6659840c84170b8ff8df6dd3daab2. Latest consensus state: current_term: 1 leader_uuid: "5bb6659840c84170b8ff8df6dd3daab2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb6659840c84170b8ff8df6dd3daab2" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34885 } } }
I20260928 06:44:55.502388 31091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:55.502656 31092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bb6659840c84170b8ff8df6dd3daab2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb6659840c84170b8ff8df6dd3daab2" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 34885 } } }
I20260928 06:44:55.502771 31092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:55.502712 31095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:55.506846 31095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:55.512042 31095 catalog_manager.cc:1347] Generated new cluster ID: 44b60dd0031843b1874e095554d2453f
I20260928 06:44:55.512102 31095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:55.526888 31095 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:55.527278 31095 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:55.536942 31095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bb6659840c84170b8ff8df6dd3daab2: Generated new TSK 0
I20260928 06:44:55.537554 31095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 06:44:55.546571 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 31013
2026-09-28T06:44:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1361 ms)
[----------] 18 tests from AuthTokenIssuingTest (24483 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:55 dist-test-slave-gtr8 krb5kdc[31112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:55 dist-test-slave-gtr8 krb5kdc[31112](info): set up 2 sockets
Sep 28 06:44:55 dist-test-slave-gtr8 krb5kdc[31112](info): commencing operation
krb5kdc: starting...
W20260928 06:44:56.519870 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:56 dist-test-slave-gtr8 krb5kdc[31112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:56.642385 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:43487
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:43487,127.8.151.253:45681,127.8.151.252:44213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:56.755105 31125 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 06:44:56.755541 31125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:56.755589 31125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:56.755658 31125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:56.760485 31125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:56.760543 31125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:56.760561 31125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:56.760577 31125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:56.760591 31125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:56.765966 31125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:43487,127.8.151.253:45681,127.8.151.252:44213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:43487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:56.767015 31125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:56.768051 31125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:56.776296 31130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:56.776376 31133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:56.776325 31131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:56.776538 31125 server_base.cc:1034] running on GCE node
I20260928 06:44:56.777115 31125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:56.778216 31125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:56.779381 31125 hybrid_clock.cc:648] HybridClock initialized: now 1790577896779360 us; error 32 us; skew 500 ppm
Sep 28 06:44:56 dist-test-slave-gtr8 krb5kdc[31112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:56.781985 31125 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:44:56.782866 31125 webserver.cc:460] Webserver started at http://127.8.151.254:41837/ using document root <none> and password file <none>
I20260928 06:44:56.783439 31125 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:56.783502 31125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:56.783702 31125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:56.785830 31125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "f987bb8b31894d0d97e75151e5b423d8"
format_stamp: "Formatted at 2026-09-28 06:44:56 on dist-test-slave-gtr8"
server_key: "5d702e1afe544c896252c0431882e6af"
server_key_iv: "a967c5039580cc958d30fd5223c12421"
server_key_version: "encryptionkey@0"
I20260928 06:44:56.786365 31125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "f987bb8b31894d0d97e75151e5b423d8"
format_stamp: "Formatted at 2026-09-28 06:44:56 on dist-test-slave-gtr8"
server_key: "5d702e1afe544c896252c0431882e6af"
server_key_iv: "a967c5039580cc958d30fd5223c12421"
server_key_version: "encryptionkey@0"
I20260928 06:44:56.790285 31125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260928 06:44:56.792879 31140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:56.794226 31125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 06:44:56.794360 31125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "f987bb8b31894d0d97e75151e5b423d8"
format_stamp: "Formatted at 2026-09-28 06:44:56 on dist-test-slave-gtr8"
server_key: "5d702e1afe544c896252c0431882e6af"
server_key_iv: "a967c5039580cc958d30fd5223c12421"
server_key_version: "encryptionkey@0"
I20260928 06:44:56.794461 31125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:56.816025 31125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:56.819111 31125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:56.819319 31125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:56.826017 31125 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:43487
I20260928 06:44:56.826290 31192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:43487 every 8 connection(s)
I20260928 06:44:56.827040 31125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:44:56.828686 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 31125
I20260928 06:44:56.828824 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:44:56.829120 8799 external_mini_cluster.cc:1428] Setting key 775a0430d47e66a34878ea6932a8cc85
I20260928 06:44:56.830188 31193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:56.834517 31193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.151.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.253@KRBTEST.COM" created.
Sep 28 06:44:56 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:56.852144 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:56.837398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:43487 (local address 127.0.0.1:55160)
0928 06:44:56.838068 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:56.838116 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:56.839011 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:56.839681 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:56.839686 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:56.840091 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:56.840328 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:56.840345 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:56.842066 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:56.842069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:56.843065 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:56.843078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:56.844168 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:56.844172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:56.844287 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:56.844921 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:56.844942 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:56.846782 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:56.848831 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:56.848842 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:56.848853 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:56.849139 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:56.849431 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:56.849434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:56.849436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:56.849573 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:56.850495 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:56.850503 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:56.850664 (+ 161us) client_negotiation.cc:769] Sending connection context
0928 06:44:56.850848 (+ 184us) client_negotiation.cc:240] Negotiation successful
0928 06:44:56.851329 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":152,"thread_start_us":146,"threads_started":1}
I20260928 06:44:56.852154 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:56.837398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55160 (local address 127.8.151.254:43487)
0928 06:44:56.837990 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:56.838008 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:56.838217 (+ 209us) server_negotiation.cc:407] Connection header received
0928 06:44:56.839329 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:56.839333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:56.839505 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:56.839664 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:56.840529 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:56.841929 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:56.843240 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:56.843807 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:56.843959 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:56.846948 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:56.846955 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:56.846968 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:56.846995 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:56.848726 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:56.849237 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:56.849241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:56.849243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:56.849344 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:56.849694 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:56.849696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:56.849697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:56.850049 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:56.850222 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:56.850926 (+ 704us) server_negotiation.cc:299] Negotiation successful
0928 06:44:56.851249 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":443,"spinlock_wait_cycles":47360,"thread_start_us":159,"threads_started":1}
I20260928 06:44:56.854410 31193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:44:56.855230 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:56.854899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45681 (local address 127.0.0.1:34342)
0928 06:44:56.855037 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:56.855106 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260928 06:44:56.855410 31143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45681: connect: Connection refused (error 111)
I20260928 06:44:56.856249 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:56.856059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45681 (local address 127.0.0.1:34358)
0928 06:44:56.856155 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:56.856199 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 06:44:56.856382 31193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.253:45681: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45681: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab.
Entry for principal kudu/127.8.151.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab.
I20260928 06:44:56.858242 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.253:45681
--webserver_interface=127.8.151.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab
--principal=kudu/127.8.151.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:43487,127.8.151.253:45681,127.8.151.252:44213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:44:56.905619 31193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } attempt: 1
I20260928 06:44:56.906396 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:56.906113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45681 (local address 127.0.0.1:34362)
0928 06:44:56.906260 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:56.906317 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 06:44:56.907121 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:56.906927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45681 (local address 127.0.0.1:34366)
0928 06:44:56.907021 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:56.907066 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 06:44:56.907320 31193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.253:45681: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45681: connect: Connection refused (error 111)
W20260928 06:44:56.972455 31200 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 06:44:56.972817 31200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:56.972919 31200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:56.972980 31200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260928 06:44:56.975471 31193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } attempt: 2
I20260928 06:44:56.976334 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:56.975996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45681 (local address 127.0.0.1:34380)
0928 06:44:56.976159 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:56.976238 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 06:44:56.977090 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:56.976886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45681 (local address 127.0.0.1:34396)
0928 06:44:56.976979 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:56.977027 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 06:44:56.977278 31193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.253:45681: Network error: Client connection negotiation failed: client connection to 127.8.151.253:45681: connect: Connection refused (error 111)
W20260928 06:44:56.977656 31200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:56.977700 31200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:56.977718 31200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:56.977749 31200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:56.977778 31200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:56.983500 31200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:43487,127.8.151.253:45681,127.8.151.252:44213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab
--principal=kudu/127.8.151.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.253:45681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.151.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:56.984639 31200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:56.985737 31200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:56.994915 31206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:56.995071 31200 server_base.cc:1034] running on GCE node
W20260928 06:44:56.994915 31205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:56.994935 31208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:56.996881 31200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:56.998099 31200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:56.999256 31200 hybrid_clock.cc:648] HybridClock initialized: now 1790577896999212 us; error 65 us; skew 500 ppm
Sep 28 06:44:56 dist-test-slave-gtr8 krb5kdc[31112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:57.001953 31200 init.cc:377] Logged in from keytab as kudu/127.8.151.253@KRBTEST.COM (short username kudu)
I20260928 06:44:57.002801 31200 webserver.cc:460] Webserver started at http://127.8.151.253:46025/ using document root <none> and password file <none>
I20260928 06:44:57.003434 31200 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:57.003515 31200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:57.003756 31200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:57.006033 31200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/data/instance:
uuid: "c4f9423391b74079aa6c25796124e465"
format_stamp: "Formatted at 2026-09-28 06:44:56 on dist-test-slave-gtr8"
server_key: "69da15f93f9b98d70c506e54388c89be"
server_key_iv: "75478d31a045693b415033ce17e55942"
server_key_version: "encryptionkey@0"
I20260928 06:44:57.006572 31200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/wal/instance:
uuid: "c4f9423391b74079aa6c25796124e465"
format_stamp: "Formatted at 2026-09-28 06:44:56 on dist-test-slave-gtr8"
server_key: "69da15f93f9b98d70c506e54388c89be"
server_key_iv: "75478d31a045693b415033ce17e55942"
server_key_version: "encryptionkey@0"
I20260928 06:44:57.010095 31200 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260928 06:44:57.013387 31215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:57.014667 31200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 06:44:57.014788 31200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/wal
uuid: "c4f9423391b74079aa6c25796124e465"
format_stamp: "Formatted at 2026-09-28 06:44:56 on dist-test-slave-gtr8"
server_key: "69da15f93f9b98d70c506e54388c89be"
server_key_iv: "75478d31a045693b415033ce17e55942"
server_key_version: "encryptionkey@0"
I20260928 06:44:57.014917 31200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:57.038245 31200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:57.041570 31200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:57.041723 31200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:57.048782 31200 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.253:45681
I20260928 06:44:57.049088 31267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.253:45681 every 8 connection(s)
I20260928 06:44:57.049710 31200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
I20260928 06:44:57.053069 31268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:57.054641 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 31200
I20260928 06:44:57.054761 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-1/wal/instance
I20260928 06:44:57.055030 8799 external_mini_cluster.cc:1428] Setting key 43f03fd315b1b2fd267a447e12a6a394
I20260928 06:44:57.058094 31268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.151.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.252@KRBTEST.COM" created.
I20260928 06:44:57.068427 31193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } attempt: 3
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:57.074937 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.062173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.8.151.254:43487)
0928 06:44:57.062312 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.062316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.062333 (+ 17us) server_negotiation.cc:407] Connection header received
0928 06:44:57.063119 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.063123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.063232 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.063334 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.064641 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.065484 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.066994 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.067296 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.067388 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.070097 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.070101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.070104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.070130 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.072552 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.073196 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.073200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.073202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.073289 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.073664 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.073668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.073669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.073867 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.073939 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.074485 (+ 546us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.074754 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 06:44:57.075716 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.061082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:43487 (local address 127.0.0.1:55162)
0928 06:44:57.061807 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.061870 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:57.062889 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:57.063508 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:57.063518 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:57.063875 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:57.064490 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.064507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.065622 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.065625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.066876 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.066886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.067450 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.067455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.067631 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.068178 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:57.068194 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:57.069931 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:57.072729 (+ 2798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.072735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.072750 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.073087 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.073393 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.073395 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.073397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.073559 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.074012 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:57.074018 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:57.074247 (+ 229us) client_negotiation.cc:769] Sending connection context
0928 06:44:57.074409 (+ 162us) client_negotiation.cc:240] Negotiation successful
0928 06:44:57.074738 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":160,"threads_started":1}
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
I20260928 06:44:57.077206 31268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:44:57.080996 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.068849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45681 (local address 127.0.0.1:34412)
0928 06:44:57.069154 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.069169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:57.069244 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:57.070194 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:57.070197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:57.070443 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:57.070564 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.070572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.071998 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.072001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.072789 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.072799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.073586 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.073590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.073693 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.074624 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:57.074636 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:57.077356 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:57.079446 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.079449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.079451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.079689 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.079953 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.079956 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.079958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.080034 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.080494 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:57.080497 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:57.080591 (+ 94us) client_negotiation.cc:769] Sending connection context
0928 06:44:57.080705 (+ 114us) client_negotiation.cc:240] Negotiation successful
0928 06:44:57.080839 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226}
I20260928 06:44:57.081316 31273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.069398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34412 (local address 127.8.151.253:45681)
0928 06:44:57.069760 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.069772 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.069798 (+ 26us) server_negotiation.cc:407] Connection header received
0928 06:44:57.069843 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.069849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.070052 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.070205 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.070696 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.071862 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.072923 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.073466 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.073867 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.077470 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.077480 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.077492 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.077517 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.079350 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.079783 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.079789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.079790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.079869 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.080113 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.080116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.080118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.080389 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.080507 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.080813 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.080943 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":146,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab.
Entry for principal kudu/127.8.151.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab.
I20260928 06:44:57.083029 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.252:44213
--webserver_interface=127.8.151.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab
--principal=kudu/127.8.151.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:43487,127.8.151.253:45681,127.8.151.252:44213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:44:57.083312 31193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:44:57.084966 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.083721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47108)
0928 06:44:57.084836 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.084893 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1019}
I20260928 06:44:57.085790 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.085504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47114)
0928 06:44:57.085682 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.085729 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260928 06:44:57.086000 31193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:44213: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
I20260928 06:44:57.090492 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.077726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45681 (local address 127.0.0.1:34422)
0928 06:44:57.077936 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.077949 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:57.078020 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:57.078473 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:57.078476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:57.078629 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:57.078750 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.078757 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.079841 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.079844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.080688 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.080699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.081379 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.081382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.081465 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.084613 (+ 3148us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:57.084628 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:57.086759 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:57.088839 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.088843 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.088845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.089084 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.089390 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.089394 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.089396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.089487 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.089922 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:57.089925 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:57.090029 (+ 104us) client_negotiation.cc:769] Sending connection context
0928 06:44:57.090141 (+ 112us) client_negotiation.cc:240] Negotiation successful
0928 06:44:57.090300 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"spinlock_wait_cycles":6400}
I20260928 06:44:57.090616 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.077915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.8.151.253:45681)
0928 06:44:57.078236 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.078245 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.078258 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:44:57.078291 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.078294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.078336 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.078665 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.078863 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.079721 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.080812 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.081184 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.081280 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.086924 (+ 5644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.086928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.086931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.086957 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.088716 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.089194 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.089198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.089200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.089293 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.089584 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.089588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.089590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.089823 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.089923 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.090205 (+ 282us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.090377 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":252,"spinlock_wait_cycles":1408,"thread_start_us":101,"threads_started":1}
I20260928 06:44:57.091647 31268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:44:57.092419 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.092130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47116)
0928 06:44:57.092260 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.092353 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260928 06:44:57.092617 31218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
I20260928 06:44:57.093528 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.093306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47122)
0928 06:44:57.093414 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.093467 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 06:44:57.093696 31268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:44213: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
I20260928 06:44:57.117161 31193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } attempt: 1
I20260928 06:44:57.117995 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.117676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47130)
0928 06:44:57.117820 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.117901 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260928 06:44:57.118726 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.118519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47144)
0928 06:44:57.118624 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.118671 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 06:44:57.118880 31193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:44213: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
I20260928 06:44:57.142973 31268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } attempt: 1
I20260928 06:44:57.143942 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.143549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47148)
0928 06:44:57.143807 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.143863 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260928 06:44:57.144582 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.144381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47164)
0928 06:44:57.144482 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.144528 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260928 06:44:57.144760 31268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:44213: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
I20260928 06:44:57.194041 31193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } attempt: 2
I20260928 06:44:57.194828 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.194546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47172)
0928 06:44:57.194686 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.194749 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 06:44:57.195609 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.195410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47180)
0928 06:44:57.195500 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.195549 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 06:44:57.195802 31193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:44213: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
I20260928 06:44:57.212929 31268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } attempt: 2
I20260928 06:44:57.213788 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.213460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47186)
0928 06:44:57.213634 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.213706 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260928 06:44:57.214565 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.214347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47188)
0928 06:44:57.214448 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.214498 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 06:44:57.214746 31268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:44213: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
W20260928 06:44:57.231637 31275 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 06:44:57.231969 31275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:57.231997 31275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:57.232033 31275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:57.236726 31275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:57.236797 31275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:57.236820 31275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:57.236840 31275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:57.236866 31275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:44:57.244620 31275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:43487,127.8.151.253:45681,127.8.151.252:44213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab
--principal=kudu/127.8.151.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.252:44213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.151.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:44:57.245980 31275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:44:57.247255 31275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:44:57.257349 31282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:57.257344 31281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:44:57.257350 31284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:44:57.257884 31275 server_base.cc:1034] running on GCE node
I20260928 06:44:57.258462 31275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:44:57.259685 31275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:44:57.260885 31275 hybrid_clock.cc:648] HybridClock initialized: now 1790577897260838 us; error 67 us; skew 500 ppm
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577897, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:44:57.264575 31275 init.cc:377] Logged in from keytab as kudu/127.8.151.252@KRBTEST.COM (short username kudu)
I20260928 06:44:57.265784 31275 webserver.cc:460] Webserver started at http://127.8.151.252:40211/ using document root <none> and password file <none>
I20260928 06:44:57.266345 31275 fs_manager.cc:359] Metadata directory not provided
I20260928 06:44:57.266425 31275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:44:57.266657 31275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:44:57.269986 31275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/data/instance:
uuid: "8c9c422bc9c44b6aadef6698d9cd37a6"
format_stamp: "Formatted at 2026-09-28 06:44:57 on dist-test-slave-gtr8"
server_key: "bd03078ccba4dca067459ed6c4d4c97a"
server_key_iv: "0ee91cfce0cd6cccc5099383691703a8"
server_key_version: "encryptionkey@0"
I20260928 06:44:57.270601 31275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/wal/instance:
uuid: "8c9c422bc9c44b6aadef6698d9cd37a6"
format_stamp: "Formatted at 2026-09-28 06:44:57 on dist-test-slave-gtr8"
server_key: "bd03078ccba4dca067459ed6c4d4c97a"
server_key_iv: "0ee91cfce0cd6cccc5099383691703a8"
server_key_version: "encryptionkey@0"
I20260928 06:44:57.275127 31275 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260928 06:44:57.277493 31291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:44:57.278764 31275 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260928 06:44:57.278869 31275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/wal
uuid: "8c9c422bc9c44b6aadef6698d9cd37a6"
format_stamp: "Formatted at 2026-09-28 06:44:57 on dist-test-slave-gtr8"
server_key: "bd03078ccba4dca067459ed6c4d4c97a"
server_key_iv: "0ee91cfce0cd6cccc5099383691703a8"
server_key_version: "encryptionkey@0"
I20260928 06:44:57.278978 31275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:44:57.294988 31193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } attempt: 3
I20260928 06:44:57.295866 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.295558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47202)
0928 06:44:57.295710 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.295773 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260928 06:44:57.296656 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.296467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47204)
0928 06:44:57.296557 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.296601 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 06:44:57.296832 31193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:44213: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
I20260928 06:44:57.303397 31275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:44:57.305907 31268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } attempt: 3
I20260928 06:44:57.306476 31275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:44:57.306694 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.306385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47214)
0928 06:44:57.306545 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.306609 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 06:44:57.306697 31275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:44:57.307483 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.307207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47218)
0928 06:44:57.307364 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.307417 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260928 06:44:57.307657 31268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:44213: Network error: Client connection negotiation failed: client connection to 127.8.151.252:44213: connect: Connection refused (error 111)
I20260928 06:44:57.313726 31275 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.252:44213
I20260928 06:44:57.314013 31343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.252:44213 every 8 connection(s)
I20260928 06:44:57.314690 31275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
I20260928 06:44:57.318190 31344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:44:57.321106 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 31275
I20260928 06:44:57.321237 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790577605970741-8799-0/minicluster-data/master-2/wal/instance
I20260928 06:44:57.321498 8799 external_mini_cluster.cc:1428] Setting key 97292da6e18ef68a4d6fb4fceefee350
I20260928 06:44:57.323129 31344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577897, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
I20260928 06:44:57.334965 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.324850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.8.151.254:43487)
0928 06:44:57.324972 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.324976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.325314 (+ 338us) server_negotiation.cc:407] Connection header received
0928 06:44:57.325539 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.325543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.325591 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.325658 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.326453 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.327272 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.328234 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.328456 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.328552 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.331192 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.331196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.331198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.331223 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.332793 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.333261 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.333265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.333267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.333357 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.333773 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.333777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.333779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.334021 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.334110 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.334679 (+ 569us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.334800 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:44:57.335996 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.324785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:43487 (local address 127.0.0.1:55170)
0928 06:44:57.325188 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.325236 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:57.325399 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:57.325720 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:57.325726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:57.326055 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:57.326299 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.326315 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.327391 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.327395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.328119 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.328128 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.328708 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.328711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.328945 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.329529 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:57.329550 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:57.331020 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:57.332892 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.332902 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.332913 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.333165 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.333473 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.333477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.333479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.333666 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.334133 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:57.334139 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:57.334399 (+ 260us) client_negotiation.cc:769] Sending connection context
0928 06:44:57.334632 (+ 233us) client_negotiation.cc:240] Negotiation successful
0928 06:44:57.334939 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":47,"thread_start_us":124,"threads_started":1}
I20260928 06:44:57.335996 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.322840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.8.151.252:44213)
0928 06:44:57.323335 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.323354 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.323397 (+ 43us) server_negotiation.cc:407] Connection header received
0928 06:44:57.324292 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.324320 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.324746 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.325072 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.326370 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.327341 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.328498 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.329046 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.329148 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.331954 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.331961 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.331974 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.331995 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.333943 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.334405 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.334410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.334412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.334487 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.334765 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.334768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.334770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.335040 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.335262 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.335645 (+ 383us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.335775 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":342,"thread_start_us":152,"threads_started":1}
I20260928 06:44:57.337360 31344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577897, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:44:57.347563 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.337839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:45681 (local address 127.0.0.1:34438)
0928 06:44:57.337927 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.337942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:57.338016 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:57.338457 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:57.338460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:57.338620 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:57.338705 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.338711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.339640 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.339644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.340161 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.340168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.340605 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.340608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.340711 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.342118 (+ 1407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:57.342133 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:57.343566 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:57.345220 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.345223 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.345225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.345405 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.345695 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.345698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.345700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.345782 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.347003 (+ 1221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:57.347010 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:57.347106 (+ 96us) client_negotiation.cc:769] Sending connection context
0928 06:44:57.347261 (+ 155us) client_negotiation.cc:240] Negotiation successful
0928 06:44:57.347391 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260928 06:44:57.347813 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.338116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34438 (local address 127.8.151.253:45681)
0928 06:44:57.338258 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.338263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.338277 (+ 14us) server_negotiation.cc:407] Connection header received
0928 06:44:57.338317 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.338320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.338367 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.338438 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.338824 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.339536 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.340266 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.340509 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.340605 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.343714 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.343717 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.343720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.343749 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.345123 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.345496 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.345500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.345502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.345590 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.345871 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.345874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.345876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.346071 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.346168 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.347337 (+ 1169us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.347463 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:44:57.348546 31344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:44:57.348863 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.338785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55176 (local address 127.8.151.254:43487)
0928 06:44:57.338901 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.338905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.338981 (+ 76us) server_negotiation.cc:407] Connection header received
0928 06:44:57.339103 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.339106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.339184 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.339258 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.339948 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.340726 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.341841 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.342127 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.342235 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.345252 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.345255 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.345258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.345283 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.346903 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.347397 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.347401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.347402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.347485 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.347924 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.347927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.347929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.348116 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.348308 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.348612 (+ 304us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.348717 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577897, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
I20260928 06:44:57.357975 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.349003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47240)
0928 06:44:57.349244 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.349257 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:57.349318 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:57.349870 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:57.349873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:57.350028 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:57.350097 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.350102 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.351084 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.351087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.351918 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.351929 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.352601 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.352604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.352677 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.353136 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:57.353159 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:57.354974 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:57.356604 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.356606 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.356608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.356821 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.357060 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.357063 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.357064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.357143 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.357512 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:57.357515 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:57.357582 (+ 67us) client_negotiation.cc:769] Sending connection context
0928 06:44:57.357681 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 06:44:57.357805 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260928 06:44:57.358080 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.349534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47240 (local address 127.8.151.252:44213)
0928 06:44:57.349635 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.349639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.349651 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:44:57.349682 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.349685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.349723 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.349791 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.350207 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.350975 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.352034 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.352400 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.352478 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.355116 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.355120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.355123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.355170 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.356489 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.356914 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.356916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.356918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.356985 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.357218 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.357221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.357222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.357398 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.357460 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.357758 (+ 298us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.357882 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 06:44:57.358793 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.349986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34440 (local address 127.8.151.253:45681)
0928 06:44:57.350086 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.350090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.350195 (+ 105us) server_negotiation.cc:407] Connection header received
0928 06:44:57.350301 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.350304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.350341 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.350400 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.350987 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.351527 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.352221 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.352443 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.352535 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.355200 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.355203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.355205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.355229 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.356771 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.357249 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.357252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.357254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.357338 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.357614 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.357617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.357619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.357819 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.358022 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.358566 (+ 544us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.358666 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 06:44:57.362063 31344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6: Bootstrap starting.
I20260928 06:44:57.362370 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:44:57.364787 31344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:57.363818 31167 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f987bb8b31894d0d97e75151e5b423d8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55176
I20260928 06:44:57.365549 31344 log.cc:824] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:57.367817 31344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6: No bootstrap required, opened a new log
I20260928 06:44:57.371730 31344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } }
I20260928 06:44:57.371970 31344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9c422bc9c44b6aadef6698d9cd37a6, State: Initialized, Role: FOLLOWER
I20260928 06:44:57.372289 31344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } }
I20260928 06:44:57.372802 31350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } } }
I20260928 06:44:57.372991 31350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:44:57.373708 31344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [sys.catalog]: configured and running, proceeding with master startup.
W20260928 06:44:57.378615 31362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:57.378701 31362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:57.377453 31275 master_runner.cc:186] Error getting master registration for 127.8.151.254:43487: OK, instance_id { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" instance_seqno: 1790577896824166 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 06:44:57.381246 31275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 06:44:57.450807 31268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } attempt: 4
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
I20260928 06:44:57.460569 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.451448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47244)
0928 06:44:57.451610 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.451628 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:57.451735 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:57.452057 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:57.452060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:57.452279 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:57.452391 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.452397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.453406 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.453410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.454253 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.454265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.454817 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.454821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.454910 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.455453 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:57.455469 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:57.457022 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:57.458850 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.458853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.458855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.459088 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.459447 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.459450 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.459452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.459538 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.460060 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:57.460063 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:57.460160 (+ 97us) client_negotiation.cc:769] Sending connection context
0928 06:44:57.460258 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 06:44:57.460390 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 06:44:57.460968 31193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } attempt: 4
I20260928 06:44:57.461339 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.451696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47244 (local address 127.8.151.252:44213)
0928 06:44:57.451819 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.451824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.451839 (+ 15us) server_negotiation.cc:407] Connection header received
0928 06:44:57.451884 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.451887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.451934 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.452014 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.452512 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.453298 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.454374 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.454713 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.454809 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.457148 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.457151 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.457154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.457182 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.458731 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.459256 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.459260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.459261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.459350 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.459632 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.459635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.459637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.459923 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.460013 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.461010 (+ 997us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.461194 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 06:44:57.464979 31268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465: Bootstrap starting.
I20260928 06:44:57.467809 31268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:57.468681 31268 log.cc:824] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465: Log is configured to *not* fsync() on all Append() calls
Sep 28 06:44:57 dist-test-slave-gtr8 krb5kdc[31112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577896, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
I20260928 06:44:57.473301 31268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465: No bootstrap required, opened a new log
I20260928 06:44:57.477121 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.461431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:44213 (local address 127.0.0.1:47252)
0928 06:44:57.461709 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0928 06:44:57.461722 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:44:57.461798 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:44:57.468055 (+ 6257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:44:57.468058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:44:57.468266 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:44:57.468391 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.468400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.469413 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.469417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.470249 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:44:57.470262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.470825 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.470828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:44:57.470920 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.471648 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:44:57.471663 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:44:57.473163 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:44:57.475514 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.475518 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.475520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.475760 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.476066 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:44:57.476069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:44:57.476071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:44:57.476156 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:44:57.476562 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:44:57.476565 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:44:57.476688 (+ 123us) client_negotiation.cc:769] Sending connection context
0928 06:44:57.476803 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 06:44:57.476945 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199}
I20260928 06:44:57.477869 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.461926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47252 (local address 127.8.151.252:44213)
0928 06:44:57.462422 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.462426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.462437 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:44:57.462467 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.462470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.462507 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.462572 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.468545 (+ 5973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.469281 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.470368 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.470708 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.470817 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.473307 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.473311 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.473314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.473341 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.475388 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.475874 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.475878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.475880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.475974 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.476257 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.476261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.476263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.476462 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.476550 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.477570 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.477702 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":158,"threads_started":1}
I20260928 06:44:57.481220 31193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8: Bootstrap starting.
I20260928 06:44:57.484050 31193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8: Neither blocks nor log segments found. Creating new log.
I20260928 06:44:57.484918 31193 log.cc:824] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8: Log is configured to *not* fsync() on all Append() calls
I20260928 06:44:57.485713 31268 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } }
I20260928 06:44:57.485963 31268 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f9423391b74079aa6c25796124e465, State: Initialized, Role: FOLLOWER
I20260928 06:44:57.486318 31268 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } }
I20260928 06:44:57.487020 31367 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } } }
I20260928 06:44:57.487403 31367 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:44:57.487790 31268 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:57.491991 31193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8: No bootstrap required, opened a new log
W20260928 06:44:57.496104 31378 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:57.496291 31378 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:57.500423 31193 raft_consensus.cc:348] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } }
I20260928 06:44:57.500664 31193 raft_consensus.cc:729] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f987bb8b31894d0d97e75151e5b423d8, State: Initialized, Role: FOLLOWER
I20260928 06:44:57.501031 31193 consensus_queue.cc:260] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } }
I20260928 06:44:57.501682 31380 sys_catalog.cc:455] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } } }
I20260928 06:44:57.501979 31380 sys_catalog.cc:458] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:44:57.502453 31193 sys_catalog.cc:564] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:44:57.502558 31167 raft_consensus.cc:482] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 06:44:57.502688 31167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:57.504119 31167 raft_consensus.cc:504] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } }
I20260928 06:44:57.505143 31167 leader_election.cc:288] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [CANDIDATE]: Term 1 election: Requested vote from peers c4f9423391b74079aa6c25796124e465 (127.8.151.253:45681), 8c9c422bc9c44b6aadef6698d9cd37a6 (127.8.151.252:44213)
I20260928 06:44:57.505985 31242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f987bb8b31894d0d97e75151e5b423d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c4f9423391b74079aa6c25796124e465"
I20260928 06:44:57.506238 31242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:57.506435 31318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f987bb8b31894d0d97e75151e5b423d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6"
I20260928 06:44:57.506693 31318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:44:57.507581 31242 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f987bb8b31894d0d97e75151e5b423d8 in term 1.
I20260928 06:44:57.508272 31318 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f987bb8b31894d0d97e75151e5b423d8 in term 1.
I20260928 06:44:57.508809 31143 leader_election.cc:302] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c9c422bc9c44b6aadef6698d9cd37a6, f987bb8b31894d0d97e75151e5b423d8; no voters:
I20260928 06:44:57.509176 31380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:44:57.509938 31380 raft_consensus.cc:686] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [term 1 LEADER]: Becoming Leader. State: Replica: f987bb8b31894d0d97e75151e5b423d8, State: Running, Role: LEADER
I20260928 06:44:57.510351 31380 consensus_queue.cc:237] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } }
I20260928 06:44:57.512462 31380 sys_catalog.cc:455] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f987bb8b31894d0d97e75151e5b423d8. Latest consensus state: current_term: 1 leader_uuid: "f987bb8b31894d0d97e75151e5b423d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } } }
I20260928 06:44:57.513340 31380 sys_catalog.cc:458] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:57.512182 31200 master_runner.cc:186] Error getting master registration for 127.8.151.254:43487: OK, instance_id { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" instance_seqno: 1790577896824166 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 06:44:57.515337 31200 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260928 06:44:57.517339 31399 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:44:57.517457 31399 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:44:57.533579 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.519562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.8.151.253:45681)
0928 06:44:57.519706 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.519711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.524650 (+ 4939us) server_negotiation.cc:407] Connection header received
0928 06:44:57.524818 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.524822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.524871 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.524921 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.525857 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.526611 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.527684 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.528018 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.528088 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.529822 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.529826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.529828 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.529855 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.531821 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.532276 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.532280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.532282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.532366 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.532653 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.532656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.532658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.532855 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.532943 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.533298 (+ 355us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.533412 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:44:57.538617 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.509799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.8.151.254:43487)
0928 06:44:57.509910 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.509915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.520161 (+ 10246us) server_negotiation.cc:407] Connection header received
0928 06:44:57.520359 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.520362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.520409 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.520461 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.521553 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.522341 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.523575 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.523908 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.523995 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.534973 (+ 10978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.534977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.534980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.535010 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.536713 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.537250 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.537254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.537256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.537341 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.537673 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.537677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.537679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.537896 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.538024 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.538340 (+ 316us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.538459 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 06:44:57.538983 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:57.520700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.8.151.252:44213)
0928 06:44:57.520813 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.520817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.527369 (+ 6552us) server_negotiation.cc:407] Connection header received
0928 06:44:57.531348 (+ 3979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.531352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.531405 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.531469 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.532253 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.532940 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.533902 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:57.534190 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:57.534257 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:57.535822 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:44:57.535826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:44:57.535828 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:44:57.535857 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:44:57.537294 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.537745 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.537748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.537749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.537822 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:44:57.538109 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:44:57.538113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:44:57.538114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:44:57.538317 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:44:57.538401 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:57.538734 (+ 333us) server_negotiation.cc:299] Negotiation successful
0928 06:44:57.538841 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260928 06:44:57.543332 31221 tablet.cc:2369] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:44:57.549300 31398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:44:57.550522 31398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:44:57.556314 31318 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [term 1 FOLLOWER]: Refusing update from remote peer f987bb8b31894d0d97e75151e5b423d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:57.556344 31242 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [term 1 FOLLOWER]: Refusing update from remote peer f987bb8b31894d0d97e75151e5b423d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:44:57.557111 31385 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:44:57.557564 31380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 06:44:57.561213 31297 tablet.cc:2369] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 06:44:57.561558 31350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f987bb8b31894d0d97e75151e5b423d8. Latest consensus state: current_term: 1 leader_uuid: "f987bb8b31894d0d97e75151e5b423d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } } }
I20260928 06:44:57.561682 31350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:44:57.561666 31367 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f987bb8b31894d0d97e75151e5b423d8. Latest consensus state: current_term: 1 leader_uuid: "f987bb8b31894d0d97e75151e5b423d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } } }
I20260928 06:44:57.561776 31367 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:44:57.569940 31380 sys_catalog.cc:455] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f987bb8b31894d0d97e75151e5b423d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } } }
I20260928 06:44:57.570081 31380 sys_catalog.cc:458] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:57.571328 31367 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f987bb8b31894d0d97e75151e5b423d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } } }
I20260928 06:44:57.571444 31367 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:44:57.581959 31404 mvcc.cc:205] Tried to move back new op lower bound from 7334207068383170560 to 7334207068206923776. Current Snapshot: MvccSnapshot[applied={T|T < 7334207068383170560}]
I20260928 06:44:57.583174 31398 catalog_manager.cc:1347] Generated new cluster ID: 3a7824c189714b7ab683f469b44340de
I20260928 06:44:57.583243 31398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:44:57.585598 31385 sys_catalog.cc:455] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f987bb8b31894d0d97e75151e5b423d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } } }
I20260928 06:44:57.585714 31385 sys_catalog.cc:458] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8 [sys.catalog]: This master's current role is: LEADER
I20260928 06:44:57.586782 31350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f987bb8b31894d0d97e75151e5b423d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f987bb8b31894d0d97e75151e5b423d8" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 43487 } } peers { permanent_uuid: "c4f9423391b74079aa6c25796124e465" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 45681 } } peers { permanent_uuid: "8c9c422bc9c44b6aadef6698d9cd37a6" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 44213 } } }
I20260928 06:44:57.586897 31350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:44:57.587278 31405 mvcc.cc:205] Tried to move back new op lower bound from 7334207068383170560 to 7334207068206923776. Current Snapshot: MvccSnapshot[applied={T|T < 7334207068383170560}]
I20260928 06:44:57.597532 31398 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:44:57.598855 31398 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:44:57.607218 31398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f987bb8b31894d0d97e75151e5b423d8: Generated new TSK 0
I20260928 06:44:57.608062 31398 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 06:44:57.636569 31197 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.634782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55194 (local address 127.8.151.254:43487)
0928 06:44:57.634901 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.634906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.635520 (+ 614us) server_negotiation.cc:407] Connection header received
0928 06:44:57.635713 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.635716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.635766 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.635817 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.636417 (+ 600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55194: BlockingRecv error: recv got EOF from 127.0.0.1:55194 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260928 06:44:57.637744 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.634734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34464 (local address 127.8.151.253:45681)
0928 06:44:57.634868 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.634873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.637036 (+ 2163us) server_negotiation.cc:407] Connection header received
0928 06:44:57.637122 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.637125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.637177 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.637265 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.637626 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34464: BlockingRecv error: recv got EOF from 127.0.0.1:34464 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260928 06:44:57.638504 31347 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.635550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.8.151.252:44213)
0928 06:44:57.635662 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.635666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.637639 (+ 1973us) server_negotiation.cc:407] Connection header received
0928 06:44:57.637801 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.637804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.637849 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.637896 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.638380 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47284: BlockingRecv error: recv got EOF from 127.0.0.1:47284 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260928 06:44:57.642129 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.640730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34474 (local address 127.8.151.253:45681)
0928 06:44:57.640835 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.640839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.641227 (+ 388us) server_negotiation.cc:407] Connection header received
0928 06:44:57.641431 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.641434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.641519 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.641568 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.642036 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34474: BlockingRecv error: recv got EOF from 127.0.0.1:34474 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260928 06:44:57.646394 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.645451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34476 (local address 127.8.151.253:45681)
0928 06:44:57.645570 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.645574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.645586 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:44:57.645761 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.645764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.645807 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.645850 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.646305 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34476: BlockingRecv error: recv got EOF from 127.0.0.1:34476 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260928 06:44:57.651470 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.650231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34484 (local address 127.8.151.253:45681)
0928 06:44:57.650336 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.650340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.650639 (+ 299us) server_negotiation.cc:407] Connection header received
0928 06:44:57.650801 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.650804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.650847 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.650893 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.651369 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34484: BlockingRecv error: recv got EOF from 127.0.0.1:34484 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260928 06:44:57.658731 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.657348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.8.151.253:45681)
0928 06:44:57.657454 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.657469 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.657883 (+ 414us) server_negotiation.cc:407] Connection header received
0928 06:44:57.658056 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.658059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.658105 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.658164 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.658638 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34486: BlockingRecv error: recv got EOF from 127.0.0.1:34486 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260928 06:44:57.670478 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.668904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34490 (local address 127.8.151.253:45681)
0928 06:44:57.669040 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.669044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.669521 (+ 477us) server_negotiation.cc:407] Connection header received
0928 06:44:57.669697 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.669700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.669754 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.669824 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.670345 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34490: BlockingRecv error: recv got EOF from 127.0.0.1:34490 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260928 06:44:57.690635 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.688921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34496 (local address 127.8.151.253:45681)
0928 06:44:57.689077 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.689082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.689635 (+ 553us) server_negotiation.cc:407] Connection header received
0928 06:44:57.689793 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.689797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.689861 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.689947 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.690509 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34496: BlockingRecv error: recv got EOF from 127.0.0.1:34496 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260928 06:44:57.726768 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.725359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.8.151.253:45681)
0928 06:44:57.725499 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.725504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.725811 (+ 307us) server_negotiation.cc:407] Connection header received
0928 06:44:57.725958 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.725961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.726025 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.726114 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.726627 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34504: BlockingRecv error: recv got EOF from 127.0.0.1:34504 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260928 06:44:57.794430 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.793208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.8.151.253:45681)
0928 06:44:57.793338 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.793344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.793527 (+ 183us) server_negotiation.cc:407] Connection header received
0928 06:44:57.793681 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.793685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.793734 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.793803 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.794304 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34516: BlockingRecv error: recv got EOF from 127.0.0.1:34516 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260928 06:44:57.925904 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:57.924799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34532 (local address 127.8.151.253:45681)
0928 06:44:57.924959 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:57.924965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:57.925046 (+ 81us) server_negotiation.cc:407] Connection header received
0928 06:44:57.925184 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:57.925187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:57.925252 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:57.925367 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:57.925771 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34532: BlockingRecv error: recv got EOF from 127.0.0.1:34532 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260928 06:44:58.185145 31274 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:44:58.184024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34548 (local address 127.8.151.253:45681)
0928 06:44:58.184188 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:58.184194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:58.184270 (+ 76us) server_negotiation.cc:407] Connection header received
0928 06:44:58.184429 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:58.184433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:58.184485 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:58.184554 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:44:58.185015 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34548: BlockingRecv error: recv got EOF from 127.0.0.1:34548 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260928 06:44:58.380162 31362 catalog_manager.cc:1259] Loaded cluster ID: 3a7824c189714b7ab683f469b44340de
I20260928 06:44:58.380266 31362 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6: loading cluster ID for follower catalog manager: success
I20260928 06:44:58.382359 31362 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6: acquiring CA information for follower catalog manager: success
I20260928 06:44:58.383765 31362 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c9c422bc9c44b6aadef6698d9cd37a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 06:44:58.498085 31378 catalog_manager.cc:1259] Loaded cluster ID: 3a7824c189714b7ab683f469b44340de
I20260928 06:44:58.498183 31378 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465: loading cluster ID for follower catalog manager: success
I20260928 06:44:58.500727 31378 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465: acquiring CA information for follower catalog manager: success
I20260928 06:44:58.502310 31378 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c4f9423391b74079aa6c25796124e465: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 06:44:58.703050 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:58.699206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34560 (local address 127.8.151.253:45681)
0928 06:44:58.699470 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:58.699475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:58.699493 (+ 18us) server_negotiation.cc:407] Connection header received
0928 06:44:58.699657 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:58.699660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:58.699786 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:58.699879 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:58.700148 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:58.700683 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:58.701541 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:58.701771 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:58.701859 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:58.702205 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:58.702662 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:58.702772 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:58.702840 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 06:44:58.702896 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":84,"threads_started":1}
I20260928 06:44:58.709928 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:58.706148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47296 (local address 127.8.151.252:44213)
0928 06:44:58.706359 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:58.706363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:58.706375 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:44:58.706450 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:58.706452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:58.706567 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:58.706656 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:58.706868 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:58.707417 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:58.708496 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:58.708733 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:58.708821 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:58.709094 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:58.709560 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:58.709649 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:58.709737 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 06:44:58.709784 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":66,"threads_started":1}
I20260928 06:44:58.716699 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:58.712837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.8.151.253:45681)
0928 06:44:58.712957 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:58.712961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:58.713183 (+ 222us) server_negotiation.cc:407] Connection header received
0928 06:44:58.713367 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:58.713370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:58.713420 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:58.713491 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:58.713712 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:58.714405 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:58.715550 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:58.715851 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:58.715918 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:58.716273 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:58.716346 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:58.716385 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:58.716529 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 06:44:58.716579 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 06:44:58.717794 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:58.713214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47312 (local address 127.8.151.252:44213)
0928 06:44:58.713312 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:58.713316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:58.713949 (+ 633us) server_negotiation.cc:407] Connection header received
0928 06:44:58.714105 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:58.714108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:58.714156 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:58.714198 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:58.714549 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:58.715310 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:58.716494 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:58.716816 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:58.716888 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:58.717337 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:58.717408 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:58.717478 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:58.717629 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 06:44:58.717680 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 06:44:58.718726 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:44:58.712863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55204 (local address 127.8.151.254:43487)
0928 06:44:58.713694 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0928 06:44:58.713699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:44:58.713981 (+ 282us) server_negotiation.cc:407] Connection header received
0928 06:44:58.714125 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:44:58.714128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:44:58.714273 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:44:58.714374 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:44:58.714700 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:58.715495 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:58.716548 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:44:58.717535 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:44:58.717633 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:44:58.718032 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:44:58.718349 (+ 317us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:44:58.718440 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 06:44:58.718529 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 06:44:58.718589 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":74,"threads_started":1}
I20260928 06:44:58.725538 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 31125
I20260928 06:44:58.731025 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 31200
I20260928 06:44:58.736076 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 31275
2026-09-28T06:44:58Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3190 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 06:44:58 dist-test-slave-gtr8 krb5kdc[31494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 06:44:58 dist-test-slave-gtr8 krb5kdc[31494](info): set up 2 sockets
Sep 28 06:44:58 dist-test-slave-gtr8 krb5kdc[31494](info): commencing operation
krb5kdc: starting...
W20260928 06:44:59.739904 8799 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/test-user.keytab.
Sep 28 06:44:59 dist-test-slave-gtr8 krb5kdc[31494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T06:44:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T06:44:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.151.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
Entry for principal kudu/127.8.151.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab.
I20260928 06:44:59.865551 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:46041
--webserver_interface=127.8.151.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:46041,127.8.151.253:36989,127.8.151.252:39755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 06:44:59.991542 31507 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 06:44:59.991819 31507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:44:59.991843 31507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:44:59.991904 31507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:44:59.996225 31507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:44:59.996285 31507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:44:59.996304 31507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:44:59.996318 31507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:44:59.996332 31507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:45:00.001658 31507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:46041,127.8.151.253:36989,127.8.151.252:39755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.254.keytab
--principal=kudu/127.8.151.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.254:46041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.151.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:45:00.002697 31507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:45:00.003732 31507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:45:00.011708 31515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:45:00.011735 31512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:45:00.011771 31513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:45:00.012220 31507 server_base.cc:1034] running on GCE node
I20260928 06:45:00.012804 31507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:45:00.014034 31507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:45:00.015204 31507 hybrid_clock.cc:648] HybridClock initialized: now 1790577900015182 us; error 34 us; skew 500 ppm
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:45:00.018317 31507 init.cc:377] Logged in from keytab as kudu/127.8.151.254@KRBTEST.COM (short username kudu)
I20260928 06:45:00.019297 31507 webserver.cc:460] Webserver started at http://127.8.151.254:38127/ using document root <none> and password file <none>
I20260928 06:45:00.019904 31507 fs_manager.cc:359] Metadata directory not provided
I20260928 06:45:00.019977 31507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:45:00.020179 31507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:45:00.022593 31507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/data/instance:
uuid: "6794a1b8c64849a18557ba37038a2a2a"
format_stamp: "Formatted at 2026-09-28 06:45:00 on dist-test-slave-gtr8"
server_key: "35f00dcb4e8a11fbd6adef4a088a30ad"
server_key_iv: "806b21a2bf6e1c2ae9562294a35b31ed"
server_key_version: "encryptionkey@0"
I20260928 06:45:00.023224 31507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/wal/instance:
uuid: "6794a1b8c64849a18557ba37038a2a2a"
format_stamp: "Formatted at 2026-09-28 06:45:00 on dist-test-slave-gtr8"
server_key: "35f00dcb4e8a11fbd6adef4a088a30ad"
server_key_iv: "806b21a2bf6e1c2ae9562294a35b31ed"
server_key_version: "encryptionkey@0"
I20260928 06:45:00.027432 31507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260928 06:45:00.030084 31522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:45:00.031446 31507 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260928 06:45:00.031558 31507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/wal
uuid: "6794a1b8c64849a18557ba37038a2a2a"
format_stamp: "Formatted at 2026-09-28 06:45:00 on dist-test-slave-gtr8"
server_key: "35f00dcb4e8a11fbd6adef4a088a30ad"
server_key_iv: "806b21a2bf6e1c2ae9562294a35b31ed"
server_key_version: "encryptionkey@0"
I20260928 06:45:00.031658 31507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:45:00.056046 31507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:45:00.059823 31507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:45:00.060010 31507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:45:00.068671 31507 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.254:46041
I20260928 06:45:00.068928 31574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.254:46041 every 8 connection(s)
I20260928 06:45:00.069866 31507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/data/info.pb
I20260928 06:45:00.072356 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 31507
I20260928 06:45:00.072497 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-0/wal/instance
I20260928 06:45:00.072818 8799 external_mini_cluster.cc:1428] Setting key 1fda27e164a03bd1fc87c56022a01a87
I20260928 06:45:00.073011 31575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:45:00.077064 31575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.151.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.253@KRBTEST.COM" created.
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:45:00.093139 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.079429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:46041 (local address 127.0.0.1:47220)
0928 06:45:00.080054 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.080105 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 06:45:00.081162 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:45:00.082014 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:45:00.082020 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:45:00.082478 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:45:00.082831 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.082852 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.084165 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.084168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.084877 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.084885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.085561 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.085564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.085674 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.086259 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:45:00.086280 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:45:00.087862 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:45:00.089853 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.089862 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.089872 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.090167 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.090527 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.090530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.090532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.090671 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.091419 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:45:00.091426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:45:00.091594 (+ 168us) client_negotiation.cc:769] Sending connection context
0928 06:45:00.091773 (+ 179us) client_negotiation.cc:240] Negotiation successful
0928 06:45:00.092183 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":93,"spinlock_wait_cycles":56576,"thread_start_us":141,"threads_started":1}
I20260928 06:45:00.093154 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.079429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.8.151.254:46041)
0928 06:45:00.080868 (+ 1439us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.080880 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.080927 (+ 47us) server_negotiation.cc:407] Connection header received
0928 06:45:00.081530 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.081536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.081790 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.081985 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.083006 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.084053 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.084981 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.085414 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.085628 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.087990 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.087995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.088012 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.088032 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.089742 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.090284 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.090290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.090292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.090425 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.090776 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.090780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.090782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.091288 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.091412 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.091850 (+ 438us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.092079 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1310,"thread_start_us":134,"threads_started":1}
I20260928 06:45:00.095597 31575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:45:00.096333 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.096071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55074)
0928 06:45:00.096181 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.096246 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260928 06:45:00.096496 31524 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
I20260928 06:45:00.097311 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.097093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55090)
0928 06:45:00.097211 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.097259 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 06:45:00.097447 31575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.253:36989: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab.
Entry for principal kudu/127.8.151.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab.
I20260928 06:45:00.102021 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.253:36989
--webserver_interface=127.8.151.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab
--principal=kudu/127.8.151.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:46041,127.8.151.253:36989,127.8.151.252:39755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:45:00.146679 31575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } attempt: 1
I20260928 06:45:00.147495 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.147217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55092)
0928 06:45:00.147351 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.147415 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 06:45:00.148202 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.147992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55106)
0928 06:45:00.148090 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.148139 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 06:45:00.148387 31575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.253:36989: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
I20260928 06:45:00.216534 31575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } attempt: 2
I20260928 06:45:00.217315 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.217011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55122)
0928 06:45:00.217165 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.217233 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 06:45:00.218005 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.217812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55130)
0928 06:45:00.217905 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.217951 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 06:45:00.218186 31575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.253:36989: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
W20260928 06:45:00.246821 31582 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 06:45:00.247095 31582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:45:00.247118 31582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:45:00.247192 31582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:45:00.251157 31582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:45:00.251225 31582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:45:00.251245 31582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:45:00.251261 31582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:45:00.251279 31582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:45:00.256526 31582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:46041,127.8.151.253:36989,127.8.151.252:39755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.253.keytab
--principal=kudu/127.8.151.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.253:36989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.151.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:45:00.257557 31582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:45:00.258531 31582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:45:00.266664 31588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:45:00.266652 31587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:45:00.266969 31590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:45:00.267549 31582 server_base.cc:1034] running on GCE node
I20260928 06:45:00.268056 31582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:45:00.269101 31582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:45:00.270285 31582 hybrid_clock.cc:648] HybridClock initialized: now 1790577900270255 us; error 49 us; skew 500 ppm
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:45:00.272861 31582 init.cc:377] Logged in from keytab as kudu/127.8.151.253@KRBTEST.COM (short username kudu)
I20260928 06:45:00.273703 31582 webserver.cc:460] Webserver started at http://127.8.151.253:41813/ using document root <none> and password file <none>
I20260928 06:45:00.274216 31582 fs_manager.cc:359] Metadata directory not provided
I20260928 06:45:00.274273 31582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:45:00.274452 31582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:45:00.276577 31582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/data/instance:
uuid: "fcc2e34bacec4b508710a9dcaf8e746c"
format_stamp: "Formatted at 2026-09-28 06:45:00 on dist-test-slave-gtr8"
server_key: "5fcb7df210d47605c0304fd9cfc357a5"
server_key_iv: "9587a86a058b563248bb26cb33ce1aad"
server_key_version: "encryptionkey@0"
I20260928 06:45:00.277279 31582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/wal/instance:
uuid: "fcc2e34bacec4b508710a9dcaf8e746c"
format_stamp: "Formatted at 2026-09-28 06:45:00 on dist-test-slave-gtr8"
server_key: "5fcb7df210d47605c0304fd9cfc357a5"
server_key_iv: "9587a86a058b563248bb26cb33ce1aad"
server_key_version: "encryptionkey@0"
I20260928 06:45:00.281172 31582 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260928 06:45:00.283825 31597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:45:00.285235 31582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260928 06:45:00.285359 31582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/wal
uuid: "fcc2e34bacec4b508710a9dcaf8e746c"
format_stamp: "Formatted at 2026-09-28 06:45:00 on dist-test-slave-gtr8"
server_key: "5fcb7df210d47605c0304fd9cfc357a5"
server_key_iv: "9587a86a058b563248bb26cb33ce1aad"
server_key_version: "encryptionkey@0"
I20260928 06:45:00.285468 31582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:45:00.309365 31575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } attempt: 3
I20260928 06:45:00.310271 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.309930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55134)
0928 06:45:00.310099 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.310175 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 06:45:00.311066 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.310826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55144)
0928 06:45:00.310935 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.310992 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 06:45:00.311317 31575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.253:36989: Network error: Client connection negotiation failed: client connection to 127.8.151.253:36989: connect: Connection refused (error 111)
I20260928 06:45:00.311398 31582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:45:00.314543 31582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:45:00.314719 31582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:45:00.322077 31582 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.253:36989
I20260928 06:45:00.322361 31649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.253:36989 every 8 connection(s)
I20260928 06:45:00.323119 31582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/data/info.pb
I20260928 06:45:00.325886 31650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:45:00.329097 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 31582
I20260928 06:45:00.329243 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-1/wal/instance
I20260928 06:45:00.329576 8799 external_mini_cluster.cc:1428] Setting key 75e157d83afe5c2fea1a65f3e5e97d8f
I20260928 06:45:00.329710 31650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.151.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.151.252@KRBTEST.COM" created.
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:45:00.347218 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.331617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.8.151.254:46041)
0928 06:45:00.331776 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.331782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.332856 (+ 1074us) server_negotiation.cc:407] Connection header received
0928 06:45:00.334119 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.334123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.334184 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.334281 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.335853 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.336693 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.338361 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.338716 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.338834 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.343199 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.343203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.343206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.343235 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.344843 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.345496 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.345499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.345500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.345582 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.345942 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.345945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.345947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.346173 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.346289 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.346889 (+ 600us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.347017 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 06:45:00.348163 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.332103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:46041 (local address 127.0.0.1:47222)
0928 06:45:00.332682 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.332744 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0928 06:45:00.333862 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:45:00.334467 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:45:00.334476 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:45:00.334954 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:45:00.335626 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.335645 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.336831 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.336834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.338238 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.338249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.338891 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.338896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.339114 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.339806 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:45:00.339824 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:45:00.342987 (+ 3163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:45:00.344978 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.344984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.345001 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.345375 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.345667 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.345670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.345672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.345843 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.346288 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:45:00.346294 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:45:00.346618 (+ 324us) client_negotiation.cc:769] Sending connection context
0928 06:45:00.346814 (+ 196us) client_negotiation.cc:240] Negotiation successful
0928 06:45:00.347176 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":147,"threads_started":1}
I20260928 06:45:00.349654 31650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
I20260928 06:45:00.359310 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.350182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55154)
0928 06:45:00.350358 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.350371 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:45:00.350444 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:45:00.350922 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:45:00.350925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:45:00.351078 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:45:00.351222 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.351230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.352608 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.352611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.353171 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.353180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.353944 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.353947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.354072 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.354486 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:45:00.354496 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:45:00.356114 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:45:00.357796 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.357800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.357802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.358035 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.358308 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.358311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.358313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.358402 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.358777 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:45:00.358780 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:45:00.358872 (+ 92us) client_negotiation.cc:769] Sending connection context
0928 06:45:00.358978 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 06:45:00.359104 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":42}
I20260928 06:45:00.359351 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.350273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55154 (local address 127.8.151.253:36989)
0928 06:45:00.350584 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.350595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.350614 (+ 19us) server_negotiation.cc:407] Connection header received
0928 06:45:00.350646 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.350650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.350797 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.350897 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.351337 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.352504 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.353281 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.353831 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.353934 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.356236 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.356244 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.356254 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.356274 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.357693 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.358125 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.358156 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.358158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.358233 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.358474 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.358476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.358477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.358693 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.358789 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.359035 (+ 246us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.359222 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":223,"thread_start_us":142,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.151.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab.
Entry for principal kudu/127.8.151.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab.
I20260928 06:45:00.361271 31650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:45:00.361790 8799 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
/tmp/dist-test-taskYzMBXE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.252:39755
--webserver_interface=127.8.151.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab
--principal=kudu/127.8.151.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.151.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.151.254:46041,127.8.151.253:36989,127.8.151.252:39755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 06:45:00.361989 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.361728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38524)
0928 06:45:00.361848 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.361928 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260928 06:45:00.362186 31600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
I20260928 06:45:00.363032 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.362827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38540)
0928 06:45:00.362922 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.362974 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 06:45:00.363229 31650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:39755: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
I20260928 06:45:00.412467 31650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } attempt: 1
I20260928 06:45:00.413398 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.413055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38544)
0928 06:45:00.413230 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.413303 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 06:45:00.414179 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.413961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38552)
0928 06:45:00.414054 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.414105 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 06:45:00.414367 31650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:39755: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
I20260928 06:45:00.454510 31575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } attempt: 4
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
I20260928 06:45:00.465025 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.455114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55170)
0928 06:45:00.455406 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.455421 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 06:45:00.455525 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:45:00.455842 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:45:00.455845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:45:00.456055 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:45:00.456192 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.456201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.457266 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.457270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.458574 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.458586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.459214 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.459218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.459323 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.459875 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:45:00.459890 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:45:00.461464 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:45:00.463416 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.463419 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.463421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.463660 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.463975 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.463978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.463980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.464075 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.464490 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:45:00.464493 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:45:00.464598 (+ 105us) client_negotiation.cc:769] Sending connection context
0928 06:45:00.464717 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 06:45:00.464853 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212}
I20260928 06:45:00.465065 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.455260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.8.151.253:36989)
0928 06:45:00.455394 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.455400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.455477 (+ 77us) server_negotiation.cc:407] Connection header received
0928 06:45:00.455650 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.455654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.455706 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.455786 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.456326 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.457115 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.458709 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.459055 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.459181 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.461620 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.461624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.461627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.461654 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.463292 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.463778 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.463782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.463784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.463879 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.464160 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.464163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.464165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.464389 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.464487 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.464784 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.464908 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 06:45:00.466413 31575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:45:00.467221 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.466935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38568)
0928 06:45:00.467056 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.467115 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 06:45:00.467994 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.467752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38574)
0928 06:45:00.467891 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.467936 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260928 06:45:00.468222 31575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:39755: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
W20260928 06:45:00.471944 31657 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 06:45:00.472224 31657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 06:45:00.472266 31657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 06:45:00.472330 31657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 06:45:00.476425 31657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 06:45:00.476481 31657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 06:45:00.476508 31657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 06:45:00.476539 31657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 06:45:00.476568 31657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 06:45:00.481990 31657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.151.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.151.254:46041,127.8.151.253:36989,127.8.151.252:39755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/krb5kdc/kudu_127.8.151.252.keytab
--principal=kudu/127.8.151.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.151.252:39755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.151.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 28 Sep 2026 06:04:39 UTC on ccycloud.quasar-pemmwb.root.comops.site
build id 4676663
I20260928 06:45:00.482515 31650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } attempt: 2
I20260928 06:45:00.483296 31657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 06:45:00.483862 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.483599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38580)
0928 06:45:00.483736 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.483791 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 06:45:00.484524 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.484329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38582)
0928 06:45:00.484425 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.484472 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 06:45:00.484614 31657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 06:45:00.484683 31650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:39755: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
W20260928 06:45:00.493780 31665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 06:45:00.493755 31662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:45:00.493970 31657 server_base.cc:1034] running on GCE node
W20260928 06:45:00.494164 31663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 06:45:00.494720 31657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 06:45:00.495847 31657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 06:45:00.496966 31657 hybrid_clock.cc:648] HybridClock initialized: now 1790577900496929 us; error 55 us; skew 500 ppm
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 06:45:00.500275 31657 init.cc:377] Logged in from keytab as kudu/127.8.151.252@KRBTEST.COM (short username kudu)
I20260928 06:45:00.501434 31657 webserver.cc:460] Webserver started at http://127.8.151.252:41461/ using document root <none> and password file <none>
I20260928 06:45:00.502111 31657 fs_manager.cc:359] Metadata directory not provided
I20260928 06:45:00.502211 31657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 06:45:00.502466 31657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 06:45:00.505257 31657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/data/instance:
uuid: "7f8218c7c9b04f218bc131570a162848"
format_stamp: "Formatted at 2026-09-28 06:45:00 on dist-test-slave-gtr8"
server_key: "0022a2cb9afdf8bad53436fa68933b01"
server_key_iv: "eeb366fe0e8e88cd1915e1d1ec852251"
server_key_version: "encryptionkey@0"
I20260928 06:45:00.505931 31657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/wal/instance:
uuid: "7f8218c7c9b04f218bc131570a162848"
format_stamp: "Formatted at 2026-09-28 06:45:00 on dist-test-slave-gtr8"
server_key: "0022a2cb9afdf8bad53436fa68933b01"
server_key_iv: "eeb366fe0e8e88cd1915e1d1ec852251"
server_key_version: "encryptionkey@0"
I20260928 06:45:00.510603 31657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260928 06:45:00.513149 31672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 06:45:00.514460 31657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260928 06:45:00.514585 31657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/data,/tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/wal
uuid: "7f8218c7c9b04f218bc131570a162848"
format_stamp: "Formatted at 2026-09-28 06:45:00 on dist-test-slave-gtr8"
server_key: "0022a2cb9afdf8bad53436fa68933b01"
server_key_iv: "eeb366fe0e8e88cd1915e1d1ec852251"
server_key_version: "encryptionkey@0"
I20260928 06:45:00.514678 31657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 06:45:00.526552 31657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 06:45:00.527367 31575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } attempt: 1
I20260928 06:45:00.528155 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.527830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38590)
0928 06:45:00.527999 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.528061 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 06:45:00.530117 31657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 06:45:00.530292 31657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 06:45:00.531073 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.530802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38598)
0928 06:45:00.530927 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.530988 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260928 06:45:00.531283 31575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.151.252:39755: Network error: Client connection negotiation failed: client connection to 127.8.151.252:39755: connect: Connection refused (error 111)
I20260928 06:45:00.537051 31657 rpc_server.cc:307] RPC server started. Bound to: 127.8.151.252:39755
I20260928 06:45:00.537479 31724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.151.252:39755 every 8 connection(s)
I20260928 06:45:00.538023 31657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/data/info.pb
I20260928 06:45:00.541492 31725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 06:45:00.546422 31725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:45:00.547818 8799 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu as pid 31657
I20260928 06:45:00.547950 8799 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYzMBXE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790577605970741-8799-0/minicluster-data/master-2/wal/instance
I20260928 06:45:00.548223 8799 external_mini_cluster.cc:1428] Setting key 2a0888e1b0d7d290ff1e1cd042b9112b
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
I20260928 06:45:00.561508 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.548693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47224 (local address 127.8.151.254:46041)
0928 06:45:00.548820 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.548825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.549749 (+ 924us) server_negotiation.cc:407] Connection header received
0928 06:45:00.550595 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.550599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.550654 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.550739 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.551816 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.552320 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.553720 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.554070 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.554191 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.557676 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.557679 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.557681 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.557705 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.559294 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.559893 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.559897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.559899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.559986 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.560343 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.560347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.560349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.560546 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.560632 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.561223 (+ 591us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.561345 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 06:45:00.562523 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.549144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.254:46041 (local address 127.0.0.1:47224)
0928 06:45:00.549599 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.549641 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 06:45:00.550396 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:45:00.550796 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:45:00.550805 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:45:00.551294 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:45:00.551647 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.551667 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.552423 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.552427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.553582 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.553592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.554228 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.554231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.554496 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.555463 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:45:00.555481 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:45:00.557521 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:45:00.559415 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.559425 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.559436 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.559776 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.560100 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.560103 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.560104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.560269 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.560655 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:45:00.560662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:45:00.560937 (+ 275us) client_negotiation.cc:769] Sending connection context
0928 06:45:00.561174 (+ 237us) client_negotiation.cc:240] Negotiation successful
0928 06:45:00.561484 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":260,"thread_start_us":122,"threads_started":1}
I20260928 06:45:00.562721 31729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.549901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38610 (local address 127.8.151.252:39755)
0928 06:45:00.550139 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.550150 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.550190 (+ 40us) server_negotiation.cc:407] Connection header received
0928 06:45:00.550411 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.550433 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.550688 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.550896 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.551780 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.553144 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.554085 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.554630 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.554732 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.558366 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.558373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.558386 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.558412 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.560173 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.560711 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.560718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.560719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.560827 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.561120 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.561123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.561125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.561418 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.561625 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.562325 (+ 700us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.562467 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":152,"thread_start_us":72,"threads_started":1}
I20260928 06:45:00.564011 31725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.254@KRBTEST.COM
I20260928 06:45:00.572746 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.564660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.253:36989 (local address 127.0.0.1:55174)
0928 06:45:00.564891 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.564905 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 06:45:00.565023 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:45:00.565370 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:45:00.565372 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:45:00.565500 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:45:00.565584 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.565589 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.566673 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.566676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.567517 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.567529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.567998 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.568002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.568101 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.568515 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:45:00.568530 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:45:00.569697 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:45:00.571221 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.571224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.571227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.571436 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.571765 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.571767 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.571769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.571841 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.572298 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:45:00.572301 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:45:00.572381 (+ 80us) client_negotiation.cc:769] Sending connection context
0928 06:45:00.572468 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 06:45:00.572590 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260928 06:45:00.572790 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.564766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55174 (local address 127.8.151.253:36989)
0928 06:45:00.564900 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.564905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.564956 (+ 51us) server_negotiation.cc:407] Connection header received
0928 06:45:00.565129 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.565133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.565184 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.565232 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.565694 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.566545 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.567647 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.567896 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.567988 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.569823 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.569826 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.569829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.569854 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.571082 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.571572 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.571576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.571577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.571674 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.571932 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.571936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.571938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.572129 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.572251 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.572538 (+ 287us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.572653 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 06:45:00.573637 31725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } has no permanent_uuid. Determining permanent_uuid...
I20260928 06:45:00.575304 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.564980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.8.151.254:46041)
0928 06:45:00.565082 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.565086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.565097 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:45:00.565180 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.565183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.565224 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.565286 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.566002 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.566747 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.567685 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.568009 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.568113 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.571040 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.571044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.571047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.571075 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.573182 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.573905 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.573909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.573910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.573985 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.574283 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.574286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.574288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.574468 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.574659 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.575011 (+ 352us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.575113 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 06:45:00.575827 31650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } attempt: 3
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.252@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.253@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.151.253@KRBTEST.COM
I20260928 06:45:00.585150 31729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.574244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.8.151.252:39755)
0928 06:45:00.574335 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.574339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.574351 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:45:00.574383 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.574386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.574425 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.574500 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.574866 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.575969 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.576933 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.578233 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.578299 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.580584 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.580587 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.580590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.580614 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.582545 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.582914 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.582917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.582919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.582989 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.583255 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.583257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.583259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.583452 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.583881 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.584640 (+ 759us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.584741 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":924,"server-negotiator.queue_time_us":31,"spinlock_wait_cycles":768}
I20260928 06:45:00.585357 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.574098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38620)
0928 06:45:00.574203 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.574215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 06:45:00.574278 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:45:00.574527 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:45:00.574530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:45:00.574677 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:45:00.574764 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.574771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.576080 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.576083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.576813 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.576824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.578383 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.578386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.578455 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.578893 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:45:00.578907 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:45:00.580251 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:45:00.582636 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.582639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.582641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.582835 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.583060 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.583062 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.583064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.583158 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.583550 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:45:00.583552 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:45:00.583631 (+ 79us) client_negotiation.cc:769] Sending connection context
0928 06:45:00.584004 (+ 373us) client_negotiation.cc:240] Negotiation successful
0928 06:45:00.584139 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260928 06:45:00.587932 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.576161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38636)
0928 06:45:00.576388 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.576401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:45:00.576469 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:45:00.576934 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:45:00.576937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:45:00.577093 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:45:00.577170 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.577177 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.578243 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.578246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.579081 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.579092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.580511 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.580515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.580597 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.581030 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:45:00.581044 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:45:00.583065 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:45:00.585406 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.585409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.585412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.585919 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.586823 (+ 904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.586827 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.586829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.586922 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.587454 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:45:00.587458 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:45:00.587550 (+ 92us) client_negotiation.cc:769] Sending connection context
0928 06:45:00.587631 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 06:45:00.587770 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260928 06:45:00.588357 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.576532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55176 (local address 127.8.151.253:36989)
0928 06:45:00.576620 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.576623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.576768 (+ 145us) server_negotiation.cc:407] Connection header received
0928 06:45:00.576918 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.576921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.576958 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.577003 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.577779 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.578448 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.579516 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.579837 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.579908 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.584151 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.584154 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.584157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.584180 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.585571 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.586251 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.586254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.586256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.586342 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.586865 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.586868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.586870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.587070 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.587314 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.587682 (+ 368us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.588212 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":401,"server-negotiator.queue_time_us":37}
I20260928 06:45:00.589443 31730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.576383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38636 (local address 127.8.151.252:39755)
0928 06:45:00.576730 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.576739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.576750 (+ 11us) server_negotiation.cc:407] Connection header received
0928 06:45:00.576779 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.576782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.576820 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.577271 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.577377 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.578130 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.579268 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.580391 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.580493 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.583397 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.583400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.583403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.583422 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.585218 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.586188 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.586192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.586224 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.586717 (+ 493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.587034 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.587037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.587039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.587336 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.589013 (+ 1677us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.589089 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.589272 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":280,"thread_start_us":137,"threads_started":1}
I20260928 06:45:00.589852 31725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848: Bootstrap starting.
I20260928 06:45:00.591792 8799 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 06:45:00.592451 31650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c: Bootstrap starting.
I20260928 06:45:00.592798 31725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848: Neither blocks nor log segments found. Creating new log.
I20260928 06:45:00.593648 31725 log.cc:824] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848: Log is configured to *not* fsync() on all Append() calls
I20260928 06:45:00.592993 31549 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6794a1b8c64849a18557ba37038a2a2a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47226
I20260928 06:45:00.594923 31650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c: Neither blocks nor log segments found. Creating new log.
I20260928 06:45:00.595821 31725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848: No bootstrap required, opened a new log
I20260928 06:45:00.595824 31650 log.cc:824] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c: Log is configured to *not* fsync() on all Append() calls
I20260928 06:45:00.598047 31650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c: No bootstrap required, opened a new log
I20260928 06:45:00.598423 31575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } attempt: 2
I20260928 06:45:00.599635 31725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } }
I20260928 06:45:00.599880 31725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8218c7c9b04f218bc131570a162848, State: Initialized, Role: FOLLOWER
I20260928 06:45:00.600196 31725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } }
I20260928 06:45:00.600687 31733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } } }
I20260928 06:45:00.601130 31733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:45:00.601521 31725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:45:00.601907 31650 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } }
I20260928 06:45:00.602131 31650 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc2e34bacec4b508710a9dcaf8e746c, State: Initialized, Role: FOLLOWER
I20260928 06:45:00.602461 31650 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } }
I20260928 06:45:00.603011 31739 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } } }
I20260928 06:45:00.603345 31739 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [sys.catalog]: This master's current role is: FOLLOWER
W20260928 06:45:00.607416 31745 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:45:00.607839 31745 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:45:00.605175 31657 master_runner.cc:186] Error getting master registration for 127.8.151.254:46041: OK, instance_id { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" instance_seqno: 1790577900066372 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 06:45:00.608266 31657 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 06:45:00.608415 31650 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [sys.catalog]: configured and running, proceeding with master startup.
Sep 28 06:45:00 dist-test-slave-gtr8 krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790577900, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.151.254@KRBTEST.COM for kudu/127.8.151.252@KRBTEST.COM
I20260928 06:45:00.613742 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.598780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.151.252:39755 (local address 127.0.0.1:38652)
0928 06:45:00.599028 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0928 06:45:00.599041 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 06:45:00.599124 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 06:45:00.604186 (+ 5062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 06:45:00.604189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 06:45:00.604397 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0928 06:45:00.604517 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.604525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.605698 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.605701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.606484 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 06:45:00.606496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.607132 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.607164 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 06:45:00.607245 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.607757 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 06:45:00.607771 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 06:45:00.609224 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 06:45:00.612138 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.612142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.612144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.612401 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.612716 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 06:45:00.612719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 06:45:00.612721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 06:45:00.612811 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 06:45:00.613257 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 06:45:00.613260 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 06:45:00.613358 (+ 98us) client_negotiation.cc:769] Sending connection context
0928 06:45:00.613440 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 06:45:00.613572 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260928 06:45:00.614393 31730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.599003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38652 (local address 127.8.151.252:39755)
0928 06:45:00.603903 (+ 4900us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.603907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.603925 (+ 18us) server_negotiation.cc:407] Connection header received
0928 06:45:00.603966 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.603969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.604015 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.604639 (+ 624us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.604735 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.605555 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.606632 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.606999 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.607485 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.609350 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.609354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.609357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.609381 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.612010 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.612524 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.612527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.612529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.612607 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.612921 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.612924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.612926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.613128 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.614030 (+ 902us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.614104 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.614237 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4815}
W20260928 06:45:00.616008 31757 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 06:45:00.616094 31757 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 06:45:00.617491 31575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a: Bootstrap starting.
I20260928 06:45:00.621213 31575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a: Neither blocks nor log segments found. Creating new log.
I20260928 06:45:00.618021 31582 master_runner.cc:186] Error getting master registration for 127.8.151.254:46041: OK, instance_id { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" instance_seqno: 1790577900066372 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 06:45:00.621788 31582 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 06:45:00.622054 31575 log.cc:824] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a: Log is configured to *not* fsync() on all Append() calls
I20260928 06:45:00.624217 31575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a: No bootstrap required, opened a new log
I20260928 06:45:00.636389 31575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } }
I20260928 06:45:00.636713 31575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6794a1b8c64849a18557ba37038a2a2a, State: Initialized, Role: FOLLOWER
I20260928 06:45:00.637233 31575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } }
I20260928 06:45:00.637813 31760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } } }
I20260928 06:45:00.638001 31760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:45:00.638785 31575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 06:45:00.638863 31549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 06:45:00.638963 31549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:45:00.640342 31549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } }
I20260928 06:45:00.641362 31549 leader_election.cc:288] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [CANDIDATE]: Term 1 election: Requested vote from peers fcc2e34bacec4b508710a9dcaf8e746c (127.8.151.253:36989), 7f8218c7c9b04f218bc131570a162848 (127.8.151.252:39755)
I20260928 06:45:00.642192 31699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6794a1b8c64849a18557ba37038a2a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7f8218c7c9b04f218bc131570a162848"
I20260928 06:45:00.642268 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6794a1b8c64849a18557ba37038a2a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fcc2e34bacec4b508710a9dcaf8e746c"
I20260928 06:45:00.642452 31699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:45:00.642520 31624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [term 0 FOLLOWER]: Advancing to term 1
I20260928 06:45:00.643831 31624 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6794a1b8c64849a18557ba37038a2a2a in term 1.
I20260928 06:45:00.644248 31699 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6794a1b8c64849a18557ba37038a2a2a in term 1.
I20260928 06:45:00.646998 31525 leader_election.cc:302] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6794a1b8c64849a18557ba37038a2a2a, 7f8218c7c9b04f218bc131570a162848; no voters:
I20260928 06:45:00.651713 31760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 06:45:00.651947 31760 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [term 1 LEADER]: Becoming Leader. State: Replica: 6794a1b8c64849a18557ba37038a2a2a, State: Running, Role: LEADER
I20260928 06:45:00.652396 31507 master_runner.cc:263] No leader master found from master 6794a1b8c64849a18557ba37038a2a2a
I20260928 06:45:00.652457 31507 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 06:45:00.652546 31760 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } }
I20260928 06:45:00.654343 31760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6794a1b8c64849a18557ba37038a2a2a. Latest consensus state: current_term: 1 leader_uuid: "6794a1b8c64849a18557ba37038a2a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } } }
I20260928 06:45:00.654449 31760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [sys.catalog]: This master's current role is: LEADER
I20260928 06:45:00.661007 31779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 06:45:00.665325 31779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 06:45:00.669307 31730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.646057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38656 (local address 127.8.151.252:39755)
0928 06:45:00.646712 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.646718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.656080 (+ 9362us) server_negotiation.cc:407] Connection header received
0928 06:45:00.656247 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.656250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.656317 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.656389 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.657248 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.658045 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.659564 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.659911 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.660008 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.661609 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.661613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.661615 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.661639 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.663219 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.663666 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.663669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.663684 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.663770 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.664054 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.664057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.664059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.664265 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.664394 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.669015 (+ 4621us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.669142 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553}
I20260928 06:45:00.674278 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.655599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.8.151.253:36989)
0928 06:45:00.655721 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.655726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.665139 (+ 9413us) server_negotiation.cc:407] Connection header received
0928 06:45:00.665324 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.665328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.665383 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.665436 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.666244 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.667016 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.668134 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.668460 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.668528 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.670355 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.670358 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.670361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.670386 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.671980 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.672865 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.672869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.672871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.672960 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.673285 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.673288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.673290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.673508 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.673620 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.673953 (+ 333us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.674072 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 06:45:00.675253 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:00.654931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47236 (local address 127.8.151.254:46041)
0928 06:45:00.655037 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.655041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.656480 (+ 1439us) server_negotiation.cc:407] Connection header received
0928 06:45:00.656664 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.656668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.656728 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.656773 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.658668 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.659461 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.661088 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:00.661382 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:00.661452 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:00.671959 (+ 10507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 06:45:00.671963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 06:45:00.671966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 06:45:00.671992 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0928 06:45:00.673434 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.673884 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.673888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.673890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.673972 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 06:45:00.674311 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 06:45:00.674314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 06:45:00.674316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 06:45:00.674524 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 06:45:00.674624 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:00.674981 (+ 357us) server_negotiation.cc:299] Negotiation successful
0928 06:45:00.675092 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 06:45:00.684259 31699 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [term 1 FOLLOWER]: Refusing update from remote peer 6794a1b8c64849a18557ba37038a2a2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:45:00.684974 31777 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:45:00.691784 31733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6794a1b8c64849a18557ba37038a2a2a. Latest consensus state: current_term: 1 leader_uuid: "6794a1b8c64849a18557ba37038a2a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } } }
I20260928 06:45:00.691929 31733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:45:00.693638 31624 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [term 1 FOLLOWER]: Refusing update from remote peer 6794a1b8c64849a18557ba37038a2a2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 06:45:00.694255 31777 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 06:45:00.698168 31739 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6794a1b8c64849a18557ba37038a2a2a. Latest consensus state: current_term: 1 leader_uuid: "6794a1b8c64849a18557ba37038a2a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } } }
I20260928 06:45:00.698290 31739 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:45:00.706715 31777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6794a1b8c64849a18557ba37038a2a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } } }
I20260928 06:45:00.706858 31777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [sys.catalog]: This master's current role is: LEADER
I20260928 06:45:00.708074 31739 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6794a1b8c64849a18557ba37038a2a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } } }
I20260928 06:45:00.708190 31739 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:45:00.708577 31783 mvcc.cc:205] Tried to move back new op lower bound from 7334207081182208000 to 7334207081076396032. Current Snapshot: MvccSnapshot[applied={T|T < 7334207081182208000}]
I20260928 06:45:00.711709 31786 mvcc.cc:205] Tried to move back new op lower bound from 7334207081182208000 to 7334207081076396032. Current Snapshot: MvccSnapshot[applied={T|T < 7334207081182208000}]
I20260928 06:45:00.713358 31760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6794a1b8c64849a18557ba37038a2a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } } }
I20260928 06:45:00.713479 31760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a [sys.catalog]: This master's current role is: LEADER
I20260928 06:45:00.714561 31733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6794a1b8c64849a18557ba37038a2a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6794a1b8c64849a18557ba37038a2a2a" member_type: VOTER last_known_addr { host: "127.8.151.254" port: 46041 } } peers { permanent_uuid: "fcc2e34bacec4b508710a9dcaf8e746c" member_type: VOTER last_known_addr { host: "127.8.151.253" port: 36989 } } peers { permanent_uuid: "7f8218c7c9b04f218bc131570a162848" member_type: VOTER last_known_addr { host: "127.8.151.252" port: 39755 } } }
I20260928 06:45:00.714689 31733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 06:45:00.715164 31784 mvcc.cc:205] Tried to move back new op lower bound from 7334207081182208000 to 7334207081076396032. Current Snapshot: MvccSnapshot[applied={T|T < 7334207081182208000}]
I20260928 06:45:00.721367 31779 catalog_manager.cc:1347] Generated new cluster ID: 862867671e684b77ab720425d850db6a
I20260928 06:45:00.721446 31779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 06:45:00.739362 31779 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 06:45:00.740751 31779 catalog_manager.cc:1504] Loading token signing keys...
I20260928 06:45:00.768018 31779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6794a1b8c64849a18557ba37038a2a2a: Generated new TSK 0
I20260928 06:45:00.768944 31779 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 06:45:00.819355 31730 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.811704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.8.151.252:39755)
0928 06:45:00.811838 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.811843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.818140 (+ 6297us) server_negotiation.cc:407] Connection header received
0928 06:45:00.818427 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.818431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.818490 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.818591 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.819220 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38670: BlockingRecv error: recv got EOF from 127.0.0.1:38670 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260928 06:45:00.820362 31578 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.814222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47252 (local address 127.8.151.254:46041)
0928 06:45:00.814353 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.814357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.819302 (+ 4945us) server_negotiation.cc:407] Connection header received
0928 06:45:00.819502 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.819506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.819564 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.819620 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.820232 (+ 612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47252: BlockingRecv error: recv got EOF from 127.0.0.1:47252 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260928 06:45:00.821453 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.818253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55186 (local address 127.8.151.253:36989)
0928 06:45:00.818364 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.818368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.820051 (+ 1683us) server_negotiation.cc:407] Connection header received
0928 06:45:00.820635 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.820639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.820695 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.820764 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.821331 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55186: BlockingRecv error: recv got EOF from 127.0.0.1:55186 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260928 06:45:00.828672 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.826852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55192 (local address 127.8.151.253:36989)
0928 06:45:00.826969 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.826973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.827623 (+ 650us) server_negotiation.cc:407] Connection header received
0928 06:45:00.827854 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.827858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.827939 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.828011 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.828563 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55192: BlockingRecv error: recv got EOF from 127.0.0.1:55192 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260928 06:45:00.834057 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.832586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.8.151.253:36989)
0928 06:45:00.832688 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.832692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.833134 (+ 442us) server_negotiation.cc:407] Connection header received
0928 06:45:00.833323 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.833326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.833379 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.833433 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.833957 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55200: BlockingRecv error: recv got EOF from 127.0.0.1:55200 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260928 06:45:00.839999 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.838445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55202 (local address 127.8.151.253:36989)
0928 06:45:00.838555 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.838559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.838976 (+ 417us) server_negotiation.cc:407] Connection header received
0928 06:45:00.839118 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.839122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.839197 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.839269 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.839893 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55202: BlockingRecv error: recv got EOF from 127.0.0.1:55202 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260928 06:45:00.847599 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.846147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55208 (local address 127.8.151.253:36989)
0928 06:45:00.846300 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.846304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.846560 (+ 256us) server_negotiation.cc:407] Connection header received
0928 06:45:00.846758 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.846762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.846817 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.846872 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.847466 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55208: BlockingRecv error: recv got EOF from 127.0.0.1:55208 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260928 06:45:00.862676 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.861083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55216 (local address 127.8.151.253:36989)
0928 06:45:00.861242 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.861248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.861540 (+ 292us) server_negotiation.cc:407] Connection header received
0928 06:45:00.861777 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.861782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.861846 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.861913 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.862545 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55216: BlockingRecv error: recv got EOF from 127.0.0.1:55216 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260928 06:45:00.882367 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.881102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.8.151.253:36989)
0928 06:45:00.881235 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.881240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.881486 (+ 246us) server_negotiation.cc:407] Connection header received
0928 06:45:00.881654 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.881657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.881712 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.881784 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.882239 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55230: BlockingRecv error: recv got EOF from 127.0.0.1:55230 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260928 06:45:00.917718 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.916507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.8.151.253:36989)
0928 06:45:00.916632 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.916636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.916855 (+ 219us) server_negotiation.cc:407] Connection header received
0928 06:45:00.917001 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.917004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.917053 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.917122 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.917579 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55246: BlockingRecv error: recv got EOF from 127.0.0.1:55246 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260928 06:45:00.984911 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:00.983738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55260 (local address 127.8.151.253:36989)
0928 06:45:00.983861 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:00.983866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:00.984033 (+ 167us) server_negotiation.cc:407] Connection header received
0928 06:45:00.984179 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:00.984181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:00.984232 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:00.984310 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:00.984790 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55260: BlockingRecv error: recv got EOF from 127.0.0.1:55260 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260928 06:45:01.116179 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:01.115011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55270 (local address 127.8.151.253:36989)
0928 06:45:01.115238 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:01.115244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:01.115303 (+ 59us) server_negotiation.cc:407] Connection header received
0928 06:45:01.115450 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:01.115454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:01.115516 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:01.115615 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:01.116050 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55270: BlockingRecv error: recv got EOF from 127.0.0.1:55270 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260928 06:45:01.375593 31655 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 06:45:01.374416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.8.151.253:36989)
0928 06:45:01.374544 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:01.374548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:01.374564 (+ 16us) server_negotiation.cc:407] Connection header received
0928 06:45:01.374771 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:01.374776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:01.374841 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:01.374940 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0928 06:45:01.375461 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55272: BlockingRecv error: recv got EOF from 127.0.0.1:55272 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 06:45:01.609428 31745 catalog_manager.cc:1259] Loaded cluster ID: 862867671e684b77ab720425d850db6a
I20260928 06:45:01.609521 31745 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848: loading cluster ID for follower catalog manager: success
I20260928 06:45:01.611939 31745 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848: acquiring CA information for follower catalog manager: success
I20260928 06:45:01.613521 31745 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7f8218c7c9b04f218bc131570a162848: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 06:45:01.617215 31757 catalog_manager.cc:1259] Loaded cluster ID: 862867671e684b77ab720425d850db6a
I20260928 06:45:01.617278 31757 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c: loading cluster ID for follower catalog manager: success
I20260928 06:45:01.618973 31757 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c: acquiring CA information for follower catalog manager: success
I20260928 06:45:01.620139 31757 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fcc2e34bacec4b508710a9dcaf8e746c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 06:45:01.893873 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:01.889794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55274 (local address 127.8.151.253:36989)
0928 06:45:01.890008 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:01.890012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:01.890099 (+ 87us) server_negotiation.cc:407] Connection header received
0928 06:45:01.890256 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:01.890259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:01.890385 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:01.890493 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:45:01.890769 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:01.891351 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:01.892240 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:01.892568 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:01.892673 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:01.892946 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:45:01.893509 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:45:01.893629 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:01.893659 (+ 30us) server_negotiation.cc:299] Negotiation successful
0928 06:45:01.893722 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
I20260928 06:45:01.900671 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:01.896887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38680 (local address 127.8.151.252:39755)
0928 06:45:01.897139 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:01.897143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:01.897156 (+ 13us) server_negotiation.cc:407] Connection header received
0928 06:45:01.897287 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:01.897290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:01.897418 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:01.897506 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:45:01.897720 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:01.898214 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:01.899328 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:01.899544 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:01.899616 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:01.899885 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:45:01.900298 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:45:01.900387 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:01.900470 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 06:45:01.900521 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":99,"threads_started":1}
I20260928 06:45:01.908157 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:01.903839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38696 (local address 127.8.151.252:39755)
0928 06:45:01.903966 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:01.903969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:01.904470 (+ 501us) server_negotiation.cc:407] Connection header received
0928 06:45:01.904516 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:01.904518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:01.904563 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:01.904637 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:45:01.904919 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:01.905653 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:01.906898 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:01.907231 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:01.907299 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:01.907636 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:45:01.907714 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:45:01.907772 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:01.907971 (+ 199us) server_negotiation.cc:299] Negotiation successful
0928 06:45:01.908027 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 06:45:01.908268 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:01.904331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.8.151.253:36989)
0928 06:45:01.904440 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:01.904444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:01.904640 (+ 196us) server_negotiation.cc:407] Connection header received
0928 06:45:01.904828 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:01.904832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:01.904888 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:01.904932 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:45:01.905279 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:01.905977 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:01.907062 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:01.907404 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:01.907468 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:01.907805 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:45:01.907902 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:45:01.907945 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:01.908101 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 06:45:01.908148 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 06:45:01.909775 31869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 06:45:01.904130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.8.151.254:46041)
0928 06:45:01.904831 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0928 06:45:01.904836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 06:45:01.904848 (+ 12us) server_negotiation.cc:407] Connection header received
0928 06:45:01.904940 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 06:45:01.904942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 06:45:01.905085 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 06:45:01.905202 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 06:45:01.905529 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:01.906266 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:01.907403 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 06:45:01.907709 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 06:45:01.907797 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 06:45:01.908219 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 06:45:01.908611 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 06:45:01.908687 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 06:45:01.909580 (+ 893us) server_negotiation.cc:299] Negotiation successful
0928 06:45:01.909640 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":66,"threads_started":1}
I20260928 06:45:01.912683 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 31507
I20260928 06:45:01.920095 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 31582
I20260928 06:45:01.925014 8799 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYzMBXE/build/debug/bin/kudu with pid 31657
2026-09-28T06:45:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3190 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6381 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (295949 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST